MoodleDocs - User contributions [en]

Development:Local customisation

2008-12-18T17:51:54Z

Poltawski: Added info about workaround for an install.xml file

== Disclaimer ==

Some of these entries are things that Penny has patches for and has not yet committed, and are under discussion. The meta bug for all these items is here: http://tracker.moodle.org/browse/MDL-17376

== General customisations ==

Moodle has been designed with extensibility in mind. There are many plug-in points available though out Moodle to allow developers add new functionality to Moodle without modifying core code.

See the [[Developer_documentation#Make_a_new_plugin|make a new plugin section of the Developer documentation page]] for the different plugin types available, and documentation on how to develop for them.

== local/ folder for 'hacky' customisations ==

Sometimes it is not possible to use the available plug-in points to make your change. In situations like this then the local folder is for you. The idea is that instead of scattering your changes throughout the code base, you put them all in a folder called 'local'. Using this folder means you won't have to deal with merging problems when you upgrade the rest of your Moodle installation.

The local folder has some of the plug-in points available which are available to other modules. Perhaps most useful the local/db/ folder can be used to make database schema changes and custom role permissions.

However, using the local folder should be absolutely the last resort. Long term, you will almost certainly find it easier to maintain your changes if you can package them up as one of the standard types of plugins.

=== Local database changes and version ===

If you need to make local database customisations that are not easily encapsulated by a block or module, Moodle does support the use of a local db upgrade script, and local version number.

This is almost exactly the same as every other db/upgrade.php and version.php except for the following points:

==== local/version.php ====

local/version.php must look like:

<code php>
$local_version = 2008121700;
</code>

==== local/db/install.xml ====

Local/ has no install.xml - only an upgrade.php. This is because often the changes that you want to make are not full tables, but just extra columns, and a local install.xml makes less sense than just upgrade.php.

If you would like to create tables using using an install.xml this can be achieved by putting something like that this in your upgrade.php file:

<code php>
$result = install_from_xmldb_file(dirname(__FILE__).'/install.xml');
</code>

==== local/db/upgrade.php ====

local/db/upgrade.php must look like:

<code php>
function xmldb_local_upgrade($oldversion) {
global $CFG, $db;

$result = true;

if ($result && $result < 2008121700) {
$result = $result && create_table($table);
}
return $result;
}
</code>

=== Local post-installation data insertion ===

In discussion - see http://tracker.moodle.org/browse/MDL-17440

=== Local capabilities ===

Just like core and modules, Moodle supports the use of a db/access.php inside local/ to define local capabilities.

The formatting is exactly the same as the other db/access.php scripts - an array keyed by capability name, containing arrays of capability data, like so:

<code php>
$local_capabilities = array(
'moodle/local:capability' => array(
'captype' => 'write',
'contextlevel' => CONTEXT_SYSTEM,
'riskbitmask' => RISK_SPAM,
),

</code>

Note that for all local capabilities you add, you'll need to add language strings.
Moodle will expect to find them in local/lang/en_utf8/local.php (eg for English)
with a key (following the above example) of local:capability

=== Local event subscriptions ===

''' Pending commit '''

It is often very helpful to be able to write custom code that subscribes to normal events that Moodle throws. It's also handy to be able to throw and catch your own custom events, as the Event API provides a very handy mechanism to do signal handling.

Local event handlers get registered at install/upgrade time just as the event handlers for modules do. To trigger an update when you add a new event handler, you must bump the local version number.

Event handlers must be defined in an array, keyed by event name, with each entry in the array information about the handler, like so:

<code php>
$handlers = array(
'some_core_event' => array( // eg 'user_created'
'handlerfile' => '/local/lib.php', // example
'handlerfunction' => 'local_user_create_handler',
'schedule' => 'cron'
)
);
</code>

=== Local admin menu items and settings ===

You can add extra configuration items to Moodle by creating a file, local/settings.php which accesses the $ADMIN variable directly and adds new items to it. This will make them appear in the site administration block on the homepage, and create the config options that administrators can change. You can also add whole new custom config pages (admin_externalpage). For example:

<code php>

$ADMIN->add('root', new admin_category($name, $title);
$ADMIN->add('foo', new admin_externalpage($name, $title, $url, $cap);

</code>

=== Local backup and restore hooks ===

In discussion - see http://tracker.moodle.org/browse/MDL-17444

=== Local course deletion hook ===

This is due to be removed, and replaced with an event.

Previously, when you emptied ('''not deleted''') a course, the notify_local_course_delete method was called, which looked for a local_delete_course method in local/lib.php. The naming of this is a little ambiguous because the course is being emptied, not deleted.

Going forwards, we aim to have two events - course_emptied and course_deleted. Support for the local_delete_course method will be removed.

See http://tracker.moodle.org/browse/MDL-17445 for more info.

=== Local my moodle overrides ===

('''pending commit''')

By default, the My Moodle page shows a course overview in the center column. Some sites might want to replace that with some custom code, or even just some static content. This is very easily accomplished by:

* Creating a local/lib.php
* Putting a function in there, called local_my_moodle.

This function will be called '''instead of'' (rather than in addition to), the default center column.

=== Local stickyblocks targets ===

('''pending commit''')

In the case where you're developing a heavily customised site, it might happen that you develop a pagetype that has the ability for people to add blocks to. In this case, you might want to also make it stickyblock enabled.

This is easily achieved by:

* Create local/lib.php
* Create a method in there called local_get_sticky_pagetypes that returns an array just like the $pagetypes array at the top of admin/stickyblocks.php.

For example:

<code php>

function local_get_sticky_pagetypes() {
return array(
'custom_pagetype' => array(
'id' => 'custom_pagetype',
'lib' => '/local/lib.php',
'name' => get_string('custom_pagetype', 'local')
)
);
}

</code>

You then must create a page that extends page_base. Don't forget that if your pagetype is stickyblock enabled, it needs to take this into account in some of its methods. For example:

<code php>

class custom_pagetype extends page_base {

// normal page type class

function url_get_path() {
global $CFG;
if (defined('ADMIN_STICKYBLOCKS')) { // admin is editing stickyblocks, not a normal page
return $CFG->wwwroot . '/admin/stickyblocks.php';
}
return '';
}

function url_get_parameters() {
global $CFG;
if (defined('ADMIN_STICKYBLOCKS')) {
return array('pt' => 'custom_pagetype');
}
}
}
</code>

and map it with page_map_class:

<code php>
page_map_class('custom_pagetype', 'custom_pagetype');
</code>

=== Local user profile view hook ===

('''pending commit''')

Sometimes, it may be desirable to do something like add extra buttons to the bottom of the user's profile page - if you've developed some custom code to perform extra actions on a user, for example. This is very easy to do.

* Create local/lib.php
* Create a function in there, local_user_view.

<code php>

function local_user_view($user, $course) {

// capability check

// extra stuff

}

</code>

=== Local language strings ===

The only place in the above list that requires an additional language file is the local capabilites. These are expected to be in lang/$lang/local.php and keyed with local:capname.

Additionally, you can of course add lang/$lang/local.php just like an extra language file and call it with the usual get_string('key', 'local', $a) syntax.

There is also the ability to create a special custom lang/{$lang}_local/ directory (eg lang/en_utf8_local/), which contains just the strings that you want to override from the main language pack. Moodle will always look there '''first''' before finding the strings in the normal language pack.

This is because, historically, people have wanted to override some strings, typically things like 'course' to become 'module', or similar, and have created custom language packs. But then this forces the users to have this custom pack selected. The use of {$lang}_local is much more lightweight.

==See also==

*[http://cvs.moodle.org/moodle/lib/locallib.php?view=markup CVS:moodle/lib/locallib.php]
*Using Moodle [http://moodle.org/mod/forum/discuss.php?d=86903 Local Customisations] forum discussion

[[Category:Developer|Local customisation]]

Development:How to apply a patch

2008-12-11T13:48:58Z

Poltawski: /* Dealing with potential problems */

==Introduction==
This page explains how you can apply a patch file. Patch is a standard format, and there are many options for how to apply one. Pick the one that is easiest for you.

Perhaps most critical is the usage of the -p flag, which tells patch about the relationship between the directory where the patch file is located and the files that will be patched. See the references below for details and DO NOT assume anything.

==Apply a Patch in Windows using [http://gnuwin32.sourceforge.net/packages/patch.htm gnuwin32]==

* Download and extract patch for windows from [http://gnuwin32.sourceforge.net/packages/patch.htm sourceforge] I placed the patch.exe binary in C:\bin

* Download and extract Moodle somewhere. eg: C:\moodle

* Download the patch file and place it in the same directory you put Moodle (C:\moodle\password-policy-17.diff)

* Open the patch file with Wordpad, and click 'File' >> 'Save as...', choose a different name for the file eg ('mynewpatch.diff') and "Save as type" >> 'Text Document - MS-DOS Format'

* Open up a command text window, and type:

cd \moodle
c:\bin\patch.exe --dry-run -p1 < mynewpatch.diff

:The number after <tt>'-p'</tt> option can vary depending on the patch file, as it depends on the way the patch file was generated. Have a look at the [http://www.rt.com/man/patch.1.html 'patch' utility manual page ] to see how the <tt>'-p'</tt> option works. You could also have a look at this [http://www.linuxtutorialblog.com/post/introduction-using-diff-and-patch-tutorial diff and patch tutorial].
* You should get an output similar to this (the names and quantity of patched files vary from patch to patch):

patching file admin/settings/security.php
patching file lang/en_utf8/admin.php
patching file lib/moodlelib.php
patching file login/change_password.php
patching file login/signup.php
patching file user/edit.php
Hunk #1 succeeded at 430 (offset 2 lines).

:At this stage the patch has not been applied. We just simulated the application (with the <tt>'--dry-run'</tt> option), to see if we are going to find any problems with it. Before explaining how to actually apply the patch, we are going to talk about what could be wrong, and how to deal with it.

===Potential problems and how to deal with them===
====Potential problems====
If everything goes well, the patch will apply cleanly and life should be good! But sometimes the patch will not apply 100% cleanly due to a version mismatch between the original files used to produce the patch file and your local files. In this case, the 'patch' command will try to apply as many changes as it can, and will emit some diagnostics describing the problems it encounters.

* If you get any <tt>'Hunk #n succeeded...'</tt> messages, the patch would have been applied correctly although at different line numbers than the original file. If we had actually applied the patch, the 'patch' command would have created an additional file for each of the files where the hunk was applied at a different offset, that would be named like the original file with the additional extension <tt>.orig</tt>.

* If you get any <tt>'Hunk #n failed...'</tt> messages, the patch would have not applied correctly. In this case the 'patch' command would have created two additional files for each of the files where the hunk was not applied correctly, called:
** <tt>original-file-name.orig</tt>    This would be the original file before the patch was applied, just like above.
** <tt>original-file-name.rej </tt>    This file would contain the hunks that could not be applied correctly, so you could inspect them.

====Dealing with potential problems====
Dealing with the first problem (the offsetted hunks) is trivial: we just need to delete the .orig files once we actually apply the patch.

In the second case (failed hunks), unless you know how to fix the failed hunks by hand, you should not apply the patch, as that would corrupt your Moodle install. If you want to apply the patch and try to fix the failed hunks by hand, you should use the <tt>'-b'</tt> option. That option automatically makes a backup of every file the patch applies to, with the <tt>.orig</tt> extension. That would allow you go back to the original files state by simply overwritting the modified files with their <tt>.orig</tt> backups.

Sometimes, there will be a large difference in line numbers since a patch was generated and the patch will not apply. You can tell patch allow larger differences in line numbers by using the fuzz option '-F' to increase the number of lines difference there can be. For example patch -F 100 would allow 100 lines difference.

===Actually applying the patch===
Now that we know what could go wrong and how to deal with it, let's see how to apply the patch. We only need to remove the <tt>'--dry-run'</tt>:

cd \moodle
c:\bin\patch.exe -p1 < mynewpatch.diff

and optionally use the <tt>'-b'</tt> option if we are going to try to fix the failed hunks by hand:

cd \moodle
c:\bin\patch.exe -b -p1 < mynewpatch.diff

==Apply a Patch in Linux using "patch"==
use something like:
patch -p1 < patchfile.diff
see [http://linux.about.com/od/commands/l/blcmdl1_patch.htm here] for more details on using Patch in Linux

==See Also==

* [[Patch]]
* [[Development:How_to_create_a_patch]]
* [http://drupal.org/node/32875 Drupal - using Cygwin in Windows to apply a patch]
* [http://drupal.org/node/60818 Drupal - how to apply a patch in Mac OS X]
* [http://moodle.org/mod/forum/discuss.php?d=61379#p286372 moodle post - using gnuwin32 to apply a patch]

[[Category:Developer]]

Development:How to apply a patch

2008-12-11T13:48:34Z

Poltawski: quick note about the fuzz option

==Introduction==
This page explains how you can apply a patch file. Patch is a standard format, and there are many options for how to apply one. Pick the one that is easiest for you.

Perhaps most critical is the usage of the -p flag, which tells patch about the relationship between the directory where the patch file is located and the files that will be patched. See the references below for details and DO NOT assume anything.

==Apply a Patch in Windows using [http://gnuwin32.sourceforge.net/packages/patch.htm gnuwin32]==

* Download and extract patch for windows from [http://gnuwin32.sourceforge.net/packages/patch.htm sourceforge] I placed the patch.exe binary in C:\bin

* Download and extract Moodle somewhere. eg: C:\moodle

* Download the patch file and place it in the same directory you put Moodle (C:\moodle\password-policy-17.diff)

* Open the patch file with Wordpad, and click 'File' >> 'Save as...', choose a different name for the file eg ('mynewpatch.diff') and "Save as type" >> 'Text Document - MS-DOS Format'

* Open up a command text window, and type:

cd \moodle
c:\bin\patch.exe --dry-run -p1 < mynewpatch.diff

:The number after <tt>'-p'</tt> option can vary depending on the patch file, as it depends on the way the patch file was generated. Have a look at the [http://www.rt.com/man/patch.1.html 'patch' utility manual page ] to see how the <tt>'-p'</tt> option works. You could also have a look at this [http://www.linuxtutorialblog.com/post/introduction-using-diff-and-patch-tutorial diff and patch tutorial].
* You should get an output similar to this (the names and quantity of patched files vary from patch to patch):

patching file admin/settings/security.php
patching file lang/en_utf8/admin.php
patching file lib/moodlelib.php
patching file login/change_password.php
patching file login/signup.php
patching file user/edit.php
Hunk #1 succeeded at 430 (offset 2 lines).

:At this stage the patch has not been applied. We just simulated the application (with the <tt>'--dry-run'</tt> option), to see if we are going to find any problems with it. Before explaining how to actually apply the patch, we are going to talk about what could be wrong, and how to deal with it.

===Potential problems and how to deal with them===
====Potential problems====
If everything goes well, the patch will apply cleanly and life should be good! But sometimes the patch will not apply 100% cleanly due to a version mismatch between the original files used to produce the patch file and your local files. In this case, the 'patch' command will try to apply as many changes as it can, and will emit some diagnostics describing the problems it encounters.

* If you get any <tt>'Hunk #n succeeded...'</tt> messages, the patch would have been applied correctly although at different line numbers than the original file. If we had actually applied the patch, the 'patch' command would have created an additional file for each of the files where the hunk was applied at a different offset, that would be named like the original file with the additional extension <tt>.orig</tt>.

* If you get any <tt>'Hunk #n failed...'</tt> messages, the patch would have not applied correctly. In this case the 'patch' command would have created two additional files for each of the files where the hunk was not applied correctly, called:
** <tt>original-file-name.orig</tt>    This would be the original file before the patch was applied, just like above.
** <tt>original-file-name.rej </tt>    This file would contain the hunks that could not be applied correctly, so you could inspect them.

====Dealing with potential problems====
Dealing with the first problem (the offsetted hunks) is trivial: we just need to delete the .orig files once we actually apply the patch.

In the second case (failed hunks), unless you know how to fix the failed hunks by hand, you should not apply the patch, as that would corrupt your Moodle install. If you want to apply the patch and try to fix the failed hunks by hand, you should use the <tt>'-b'</tt> option. That option automatically makes a backup of every file the patch applies to, with the <tt>.orig</tt> extension. That would allow you go back to the original files state by simply overwritting the modified files with their <tt>.orig</tt> backups.

Sometimes, there will be a large difference in line numbers since a patch was generated and the patch will not apply. You can tell patch allow larger differences in line numbers by using the fuzz option to increase the number of lines difference there can be. For example patch -F 100 would allow 100 lines difference.

===Actually applying the patch===
Now that we know what could go wrong and how to deal with it, let's see how to apply the patch. We only need to remove the <tt>'--dry-run'</tt>:

cd \moodle
c:\bin\patch.exe -p1 < mynewpatch.diff

and optionally use the <tt>'-b'</tt> option if we are going to try to fix the failed hunks by hand:

cd \moodle
c:\bin\patch.exe -b -p1 < mynewpatch.diff

==Apply a Patch in Linux using "patch"==
use something like:
patch -p1 < patchfile.diff
see [http://linux.about.com/od/commands/l/blcmdl1_patch.htm here] for more details on using Patch in Linux

==See Also==

* [[Patch]]
* [[Development:How_to_create_a_patch]]
* [http://drupal.org/node/32875 Drupal - using Cygwin in Windows to apply a patch]
* [http://drupal.org/node/60818 Drupal - how to apply a patch in Mac OS X]
* [http://moodle.org/mod/forum/discuss.php?d=61379#p286372 moodle post - using gnuwin32 to apply a patch]

[[Category:Developer]]

Development:Enrolment plugins 2.0

2008-11-26T11:45:05Z

Poltawski: added a 'global groups' use case

{{stub}}

{{Moodle 2.0}}
The enrolment plugin system in Moodle has evolved over a long period spanning the time before and after the configurable roles system was introduced in Moodle 1.7. Despite the evolution, it is showing its age and it is probably time for a more fundamental rethink.

This page is a tool to help that thinking. There is no guarantee that anything will happen for Moodle 2.0. There are plenty of other things on the roadmap with higher priority.

==Use cases==

# [Conceptual] In all sorts of places, we need to distinguish people who can get into the course and look around (guests, admins, inspectors, ...) from people who are actually participants in the course (students and teachers).
# [Conceptual] We need to distinguish users in the course who appear in the gradebook, or whose results appear in the quiz reports.
# Teacher wants each student to pay $1000 through some billing system before they can join a course.
# Teacher wants student to enter a password before they can join a course, and depending which password they enter, they may get put in different groups.
# Teacher wants to allow students to self-register for a course, but only within a certain time period.
# Teacher wants students to self-register in Calculus 201, but only if they have completed Maths 101.
# Teacher wants to remove student's access at the end of the course, but still wants to be able to go back later and look at the gradebook data.
# Teacher wants a manual way to to unenrol a user, even if they have self-registered, and prevent them re-registering.
# If the teacher mis-clicks and unenrols the wrong user, they want to be able to immediately re-enrol them, without any data being lost.
# Teacher wants a way to automatically unenrol users who do not actively participate for 30 days.
# Admin wants to automatically synchronise enrolments with some external database or application. (While still having some manually created role assignments in the system.)
# Admin does not want teachers to be able to unenrol students that have been added by the synchronise plugins, but does want them to be able to remove students they have added manually.
# Metacourses - automatically enroling in one course should enrol you in a parent course - do we need to keep this working the way it does now? or would an alternative solution be acceptable?
# Admin wants anyone who is a teacher in any course to automatically get a special role at site level, so teachers in any course can get some site-level capabilities.
# When backing up and restoring a course, need some control which enrolments are included. (Need specific details.)
# Information about enrolment must automatically appear in course listings (but only to unenroled users?) (Do we really need this automatically generated, can't teachers put whatever they like in the coures description?)
# [Technical] When analysing data to generate reports like this, we need to be able to do a simple join in SQL to work out which users should be included in the report.
# [Technical] Data specific to certain enrolment methods (e.g. price, password) should not be stored in the course table.
# [Technical] Some of this need to be configured at the site level, others at course level. (And would we ever want things to happen at category or activity level? What about automatic enrolment of parents?)
# [Technical] Whether guests or logged in users are allowed into a course as guests should really only be a matter of setting overrides on the respective role. This should not be stored separately (irrespective of what the interface for setting these options are).
# [Technical] Configuring enrolment options for your course will either by on the course settings page, or on a separate tab next to Course settings and Assign roles.
# Teacher wants to assign a 'group of students' at a time (this group would exist at global context and might be populated automatically by an external system)
(Any role mentioned in the above text is mentioned informally, by way of motivating the feature. In the actual system, each feature should work with any role.)

==Overview of the solution==

My (Tim's) proposed solution would be this (in fact, I don't think it is a very big change from what we have now):

* The current state of who is enrolled in which course is stored indirectly via role assignments (as at present).

* A user is deemed to be a participant in a course if they have the (new) capability moodle/course:participate in that course context. If a user does not have this capability, then they may still be allowed into the course if they have the (existing) moodle/course:view capability. By default, Student, Teacher and Non-editing teachers would have the participate capability. Admin, Course creator and Guest? would get course:view.

* A user would appear in the gradebook if they have the (new) capability moodle/course:begraded. By default, only Students would have this capability.

* Enrolment plugins are just ways of creating (or deleting) role assignments, and hence letting people into courses.

* There should be hooks, when a user tries to get into a course they are not a participant of, for enrolment plugins to either let the user in silently by creating a role assignment, or by displaying some UI (like ask for a password or cash), or giving the option to go in as a guest.

* Alternatively, or as well, enrolment plugins can do things on cron, or when triggered by the events API.

* Enrolment plugins will have site-wide configuration, so administrators can control which plugins are active for the site, and set their options.

* Enrolment plugins will also have course-specific configuration, to control which plugin(s) apply to that course, and set any per-course options.

* To help with reporting and performance, we will make a new table course_participants_cache with columns id, contextid, userid, isparticipant, isgraded. That caches, for each context, or perhaps just each course context and each user, the results of has_capability for the two capabilities moodle/course:participate and moodle/course:begraded (A row will only be included if either isparticipant or isgraded is true.) This is automatically kept up to date whenever roles information changes. (Should that be contextid or courseid?).

* Extra information will be displayed on the manual Assign roles page and in the Participants list, so you can see which enrolment plugin was responsible for enrolling each student, and information about timed enrolments. (So, for example, you don't end up unenrolling a student who has paid a lot of money to enrol themselves.)

==Detailed design==

Will only be done later, if we get agreement on the use cases and solution overview.

==List of related tracker issues==

Petr offered to make a list, to help us ensure that we have considered all the things that caused problems in the past.

* ...

==See also==

* [[Roadmap|Moodle 2.0 roadmap]]

[[Category:Developer]]
[[Category:Enrolment]]

Developer meeting November 2008

2008-11-12T12:19:32Z

Poltawski: added gsoc mentor summit item

[[Developer meetings]] > November 2008 meeting

*Date: 23:00 UTC on Monday, 24 November 2008 - see the [http://moodle.org/calendar/view.php?view=day&course=5&cal_d=25&cal_m=11&cal_y=2008#event_4805 Moodle Developer Meeting calendar entry] for conversion into your time zone
*Location: [http://elluminate.remote-learner.net/join_meeting.html?meetingId=1176914355734 Elluminate Moodle Developers Meeting Room]

Everyone is welcome, though [http://moodle.org/mod/cvsadmin/view.php?id=7134 developers with CVS write access] will be given preference if the room becomes full.

''Please add items to the agenda!''

==Moodle 2.0==

* [[Development:Enrolment_plugins_2.0|Enrolment plugins proposal]]
* [[Development:Web_services|New web services system]]
* [[Development:Roles_administration_improvements_for_Moodle_2.0|New roles interfaces demo]], as an example of good JavaScript practices

==Other news==

*Google Highly Open Participation Contest (GHOP) (Helen)
*GSOC Mentor Summit - a few words about the summit (DanP & maybe Anthony/David Horat if they have anything to add)

==See also==
*[[Roadmap]]

[[Category:Developer]]

Development:Git tips

2008-10-02T12:58:23Z

Poltawski: added work in progress note

{{Work in progress}}
Many developers find git a useful tool to help them with moodle development, here some tips and workflows can be shared to help others.

== Thorough Resolved Bug QA review with git ==

The power of having fast access to all of moodle commit history means we can search and checkout any point in moodle source code history which allows us to powerfully QA a bug.

Please note this workflow might not be ideal for all scenarios (for example if lots of work has been done in the same area since).

Example Workflow to QA MDL-16600:

First checkout a new branch for us to work on testing the bug in MOODLE_19_STABLE. We can delete this branch later, its less dangerous than playing any of our 'live' branches.

<pre>
$ git checkout -b QA-MDL-16600 origin/MOODLE_19_STABLE
Branch QA-MDL-16600 set up to track remote branch refs/remotes/origin/MOODLE_19_STABLE.
Switched to a new branch "QA-MDL-16600"
</pre>

Now search for any commits related to this bug fix:
<pre>
$ git log --grep='MDL-16600'
commit 1c2c8b09469ac27a3829e7267f399356a05b9810
Author: tjhunt <tjhunt>
Date: Fri Sep 26 05:49:06 2008 +0000

MDL-16600 forcedownload broken for file resources.
</pre>

In order to test that these commits fixed the bug, we will revert this commit, and then try to reproduce the reported bug:

<pre>
$ git revert -n 1c2c8b09469ac27a3829e7267f399356a05b9810
</pre>

Once we've reproduced the problem, we will reset our repository back to HEAD:
<pre>
git reset --hard HEAD
</pre>

Now if the bug is fixed, it can be closed and we can delete our branch:
<pre>
git branch -D QA-MDL-16600
</pre>

(Please add your own tips here!)

==See Also==
[[Using GIT to backup moodledata]]

[[Category:Developer|Git tips]]
[[Category:Developer tools]]

Development:Tracking Moodle CVS with git

2008-10-02T12:48:07Z

Poltawski: added devloper tools category

Note: Work in progress. These are notes for users that are comfortable with CVS and other SCMs, doing branching, merging, etc.

==Introduction==

If you plan on doing complicated or sustained development on Moodle, you will benefit from a local revision control system. Centralized systems like [http://www.nongnu.org/cvs/ CVS] and [http://subversion.tigris.org/ SVN] have limited capabilities for tracking vendor branches. For the ultimate experience you will want to use a system with distributed capabilities like [http://git.or.cz/ git]. Alternative tools for this include [http://www.selenic.com/mercurial/ Mercurial], [http://www.darcs.net/ Darcs] and [http://svk.elixus.org/ SVK].

GIT was developed by Linus Torvalds specifically for the Linux Kernel team. It is fast, fast, fast. Its usage is slightly different from CVS/SVN, but, if kernel developers can handle it ''Moodle developers will find it '''easy''' '' ;-)

Git is packaged for most major operating systems and an up to date list of packages can be found on [http://git.or.cz/ git.or.gz].

==Obtaining git==

You'll want:
* [http://git.or.cz/ Git] (and its deps). It includes '''gitk''', a very good UI for visualising project history.

You might also find the additional git porcelain (frontend) useful:
* [http://sourceforge.net/projects/qgit qgit] a nice GUI to view the project history, make commits, etc. Recommended! Note: qgit is tricky to compile by hand, get a .deb or an .rpm
* [http://www.procode.org/stgit/ stgit] (StackedGIT is mainly for users doing heavy cherry picking. Only recommended for very advanced SCM users.)

Git is still developing very quickly, but good mature versions are already available in packaged format. This guide is written with GIT v1.5.3 in mind.

== Downloading Moodle CVS History for git ==

There are currently two methods of retrieving Moodle CVS history for git:
* Clone the [http://www.catalyst.net.nz Catalyst IT] git import found at [http://git.catalyst.net.nz/gw?p=moodle-r2.git;a=summary git://git.catalyst.net.nz/moodle-r2.git]
* Run a local cvs import into git. ( Instructions for this can be found in [[Development:Importing Moodle CVS history into git]] )

It is '''highly advisable''' to clone the Catalyst git repository as this is updated hourly and the imported CVS history will share common commit hashes allowing developers to merge each others changes easily.

==Creating a Working Copy==

In order to use your repository you must clone yourself a working copy using [http://www.kernel.org/pub/software/scm/git/docs/git-clone.html git clone].

(Let's assume your destination directory is ~/src/moodle)

git clone git://git.catalyst.net.nz/moodle-r2.git ~/src/moodle

This will clone the catalyst moodle git repostitory into a local working copy that you may make changes to and commit to. This initial clone will take some time, but note that once you have cloned this repository, you will have the complete moodle source code history locally and can easily browse and checkout any commit in moodle history without a network connection.

==Using git effectively with Moodle==

Git can be used to intelligently track changes for creating client installs based off your own local, base, customisations which in turn tracks changes to the main Moodle CVS.

If you have a local git copy of the Moodle CVS repository you will have a number of heads such as: '''MOODLE_15_STABLE''', '''MOODLE_16_STABLE''', and so on. If you wish to create a new branch for your local customisations, you can base it off an existing HEAD (one of the CVS branches that exist in the main Moodle repository). If you wish to create a new local branch, '''mymoodle''', based off the current stable 1.6 code you would:

cd ~/src/moodle
git branch mymoodle origin/MOODLE_16_STABLE
git branch ''# lists all the current branches, should show '''mymoodle'''
git checkout mymoodle ''# you are now working on the '''mymoodle''' branch''
''# add some new files (i.e. themes, blocks, etc.)''
git add .
git status ''# Will list your uncommited files''
git commit -m "Added base customisations."

To return to the '''master''' HEAD, equivalent to the main Moodle CVS HEAD branch, you would have to use:

git branch master

To get updated from cvs/catalyst

cd ~/src/moodle
git fetch # fetches changes from upstream repostiroy
git checkout mymoodle
git merge origin/MOODLE_16_STABLE # merges changes from upstream

If you wanted to use this custom local branch to create a new client install, you could use:

git clone ~/src/moodle /home/someclient/public_html/
cd /home/someclient/public_html/
git checkout mymoodle origin/mymoodle

This would create a new git repository from your branch that could be used to do a client install which can, in turn, have the client's custom files added to it. This entire process can be automated so that your '''mymoodle''' branch is tracking upstream changes from the Moodle CVS ('''MOODLE_16_STABLE''' in this example) and your client installs are tracking changes against your local branch.

==Merging a patch from GIT into CVS==

If you have cvs access, you can use the [http://www.kernel.org/pub/software/scm/git/docs/git-cvsexportcommit.html git-cvsexportcommit] command to merge or cherry pick individual patches into ''upstream''.

::git-cvsexportcommit was initially written to do merges of NZOSVLE bugfixes into Moodle. Several tools and patches to GIT and Cogito come from the NZOSVLE team. [[User:Martin Langhoff|Martin Langhoff]] 14:55, 19 July 2006 (WST)

If the patches merge cleanly, future updates will recognise the already-applied patch. In that sense the merger in git is smarter than the merger in Cogito, consider using git-pull rather than cg-update. Note that if you see conflicts using git-pull you will have to work a bit harder to resolve them. Make sure you are familiar with handling merge conflicts with pure git before trying this.

==Preparing to merge large patch series or porting over to the next Moodle release==

When you have a large set of patches to apply to CVS, or a large set of custom patches to apply on top of the next release of Moodle, you can use [http://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html git-format-patch] to see what are your ''unmerged'' patches. git-format-patch tries hard to spot already-merged patches and skip them.

If you are applying patches to CVS, you can then use git-cvsexportcommit or plain old patch -p1 filename. If you are applying them to a new git branch (as you would to merge them on top of a new Moodle release) then you want to be on that new branch, and use git-am -3 -k filename.

If you are using git-am, make sure you learn how to deal with merge conflicts, and the use of git-update-index.

==Git Tips==
See [[Development:Git tips]] for git tips to help you with moodle development.

==Helpful Documentation==

* [http://www.kernel.org/pub/software/scm/git/docs/gitcvs-migration.html git for CVS users]
* [http://www.kernel.org/pub/software/scm/git/docs/ git man pages]
* [http://vger.kernel.org/vger-lists.html The Git mailing list]
* [http://git.or.cz/gitwiki GitWiki]
* [http://www.kernel.org/git/ kernel.org Gitweb interface] (Browse the Git and associated tools' repositories)
* [http://wiki.sourcemage.org/Git_Guide Straightforward Git guide] (for Git >= 1.5)

==Moodle Forum Discussions==

* [http://moodle.org/mod/forum/discuss.php?d=91998 Tracking Moodle CVS with git (take2)]
* [http://moodle.org/mod/forum/discuss.php?d=42275 Tracking Moodle CVS with git]
* [http://moodle.org/mod/forum/discuss.php?d=34472 Merging Custom Moodle Code With Stable Releases]
* [http://moodle.org/mod/forum/discuss.php?d=66412 Moving to git, need serious help]
* [http://moodle.org/mod/forum/discuss.php?d=99763 The continuing story of moving Moodle towards Git]

[[Category:Developer|Tracking Moodle CVS with git]]
[[Category:Developer tools]]

Development:Git tips

2008-10-02T12:47:19Z

Poltawski: developer tools category

Many developers find git a useful tool to help them with moodle development, here some tips and workflows can be shared to help others.

== Thorough Resolved Bug QA review with git ==

The power of having fast access to all of moodle commit history means we can search and checkout any point in moodle source code history which allows us to powerfully QA a bug.

Please note this workflow might not be ideal for all scenarios (for example if lots of work has been done in the same area since).

Example Workflow to QA MDL-16600:

First checkout a new branch for us to work on testing the bug in MOODLE_19_STABLE. We can delete this branch later, its less dangerous than playing any of our 'live' branches.

<pre>
$ git checkout -b QA-MDL-16600 origin/MOODLE_19_STABLE
Branch QA-MDL-16600 set up to track remote branch refs/remotes/origin/MOODLE_19_STABLE.
Switched to a new branch "QA-MDL-16600"
</pre>

Now search for any commits related to this bug fix:
<pre>
$ git log --grep='MDL-16600'
commit 1c2c8b09469ac27a3829e7267f399356a05b9810
Author: tjhunt <tjhunt>
Date: Fri Sep 26 05:49:06 2008 +0000

MDL-16600 forcedownload broken for file resources.
</pre>

In order to test that these commits fixed the bug, we will revert this commit, and then try to reproduce the reported bug:

<pre>
$ git revert -n 1c2c8b09469ac27a3829e7267f399356a05b9810
</pre>

Once we've reproduced the problem, we will reset our repository back to HEAD:
<pre>
git reset --hard HEAD
</pre>

Now if the bug is fixed, it can be closed and we can delete our branch:
<pre>
git branch -D QA-MDL-16600
</pre>

==See Also==
[[Using GIT to backup moodledata]]

[[Category:Developer|Git tips]]
[[Category:Developer tools]]

Development:Git tips

2008-10-02T12:31:52Z

Poltawski: added seelaso

Development:Git tips

2008-10-02T12:09:17Z

Poltawski: info about deleting branch

Development:Git tips

2008-10-02T11:59:44Z

Poltawski: added a tip which allows QA

Development:Git tips

2008-10-02T11:29:38Z

Poltawski: updated cateogory

Many developers find git a useful tool to help them with moodle development, here some tips and workflows can be shared to help others.

== Streamlining Resolved Bug QA review with git ==

The power of having fast access to all of moodle commit history means we can search and checkout any point in moodle source code history which allows us to powerfully QA a bug.

Example Workflow to QA

[[Category:Developer|Git tips]]

Development:Git tips

2008-10-02T11:21:45Z

Poltawski: Adding new git tips page to share workflows and useful tips

Many developers find git a useful tool to help them with moodle development, here some tips and workflows can be shared to help others.

[[Category:Developer|Tracking Moodle CVS with git]]

Development:Tracking Moodle CVS with git

2008-10-02T11:19:26Z

Poltawski: fix missing link

Development:Tracking Moodle CVS with git

2008-10-02T11:19:08Z

Poltawski:

Note: Work in progress. These are notes for users that are comfortable with CVS and other SCMs, doing branching, merging, etc.

==Introduction==

If you plan on doing complicated or sustained development on Moodle, you will benefit from a local revision control system. Centralized systems like [http://www.nongnu.org/cvs/ CVS] and [http://subversion.tigris.org/ SVN] have limited capabilities for tracking vendor branches. For the ultimate experience you will want to use a system with distributed capabilities like [http://git.or.cz/ git]. Alternative tools for this include [http://www.selenic.com/mercurial/ Mercurial], [http://www.darcs.net/ Darcs] and [http://svk.elixus.org/ SVK].

GIT was developed by Linus Torvalds specifically for the Linux Kernel team. It is fast, fast, fast. Its usage is slightly different from CVS/SVN, but, if kernel developers can handle it ''Moodle developers will find it '''easy''' '' ;-)

Git is packaged for most major operating systems and an up to date list of packages can be found on [http://git.or.cz/ git.or.gz].

==Obtaining git==

You'll want:
* [http://git.or.cz/ Git] (and its deps). It includes '''gitk''', a very good UI for visualising project history.

You might also find the additional git porcelain (frontend) useful:
* [http://sourceforge.net/projects/qgit qgit] a nice GUI to view the project history, make commits, etc. Recommended! Note: qgit is tricky to compile by hand, get a .deb or an .rpm
* [http://www.procode.org/stgit/ stgit] (StackedGIT is mainly for users doing heavy cherry picking. Only recommended for very advanced SCM users.)

Git is still developing very quickly, but good mature versions are already available in packaged format. This guide is written with GIT v1.5.3 in mind.

== Downloading Moodle CVS History for git ==

There are currently two methods of retrieving Moodle CVS history for git:
* Clone the [http://www.catalyst.net.nz Catalyst IT] git import found at [http://git.catalyst.net.nz/gw?p=moodle-r2.git;a=summary git://git.catalyst.net.nz/moodle-r2.git]
* Run a local cvs import into git. ( Instructions for this can be found in [[Development:Importing Moodle CVS history into git]] )

It is '''highly advisable''' to clone the Catalyst git repository as this is updated hourly and the imported CVS history will share common commit hashes allowing developers to merge each others changes easily.

==Creating a Working Copy==

In order to use your repository you must clone yourself a working copy using [http://www.kernel.org/pub/software/scm/git/docs/git-clone.html git clone].

(Let's assume your destination directory is ~/src/moodle)

git clone git://git.catalyst.net.nz/moodle-r2.git ~/src/moodle

This will clone the catalyst moodle git repostitory into a local working copy that you may make changes to and commit to. This initial clone will take some time, but note that once you have cloned this repository, you will have the complete moodle source code history locally and can easily browse and checkout any commit in moodle history without a network connection.

==Using git effectively with Moodle==

Git can be used to intelligently track changes for creating client installs based off your own local, base, customisations which in turn tracks changes to the main Moodle CVS.

If you have a local git copy of the Moodle CVS repository you will have a number of heads such as: '''MOODLE_15_STABLE''', '''MOODLE_16_STABLE''', and so on. If you wish to create a new branch for your local customisations, you can base it off an existing HEAD (one of the CVS branches that exist in the main Moodle repository). If you wish to create a new local branch, '''mymoodle''', based off the current stable 1.6 code you would:

cd ~/src/moodle
git branch mymoodle origin/MOODLE_16_STABLE
git branch ''# lists all the current branches, should show '''mymoodle'''
git checkout mymoodle ''# you are now working on the '''mymoodle''' branch''
''# add some new files (i.e. themes, blocks, etc.)''
git add .
git status ''# Will list your uncommited files''
git commit -m "Added base customisations."

To return to the '''master''' HEAD, equivalent to the main Moodle CVS HEAD branch, you would have to use:

git branch master

To get updated from cvs/catalyst

cd ~/src/moodle
git fetch # fetches changes from upstream repostiroy
git checkout mymoodle
git merge origin/MOODLE_16_STABLE # merges changes from upstream

If you wanted to use this custom local branch to create a new client install, you could use:

git clone ~/src/moodle /home/someclient/public_html/
cd /home/someclient/public_html/
git checkout mymoodle origin/mymoodle

This would create a new git repository from your branch that could be used to do a client install which can, in turn, have the client's custom files added to it. This entire process can be automated so that your '''mymoodle''' branch is tracking upstream changes from the Moodle CVS ('''MOODLE_16_STABLE''' in this example) and your client installs are tracking changes against your local branch.

==Merging a patch from GIT into CVS==

If you have cvs access, you can use the [http://www.kernel.org/pub/software/scm/git/docs/git-cvsexportcommit.html git-cvsexportcommit] command to merge or cherry pick individual patches into ''upstream''.

::git-cvsexportcommit was initially written to do merges of NZOSVLE bugfixes into Moodle. Several tools and patches to GIT and Cogito come from the NZOSVLE team. [[User:Martin Langhoff|Martin Langhoff]] 14:55, 19 July 2006 (WST)

If the patches merge cleanly, future updates will recognise the already-applied patch. In that sense the merger in git is smarter than the merger in Cogito, consider using git-pull rather than cg-update. Note that if you see conflicts using git-pull you will have to work a bit harder to resolve them. Make sure you are familiar with handling merge conflicts with pure git before trying this.

==Preparing to merge large patch series or porting over to the next Moodle release==

When you have a large set of patches to apply to CVS, or a large set of custom patches to apply on top of the next release of Moodle, you can use [http://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html git-format-patch] to see what are your ''unmerged'' patches. git-format-patch tries hard to spot already-merged patches and skip them.

If you are applying patches to CVS, you can then use git-cvsexportcommit or plain old patch -p1 filename. If you are applying them to a new git branch (as you would to merge them on top of a new Moodle release) then you want to be on that new branch, and use git-am -3 -k filename.

If you are using git-am, make sure you learn how to deal with merge conflicts, and the use of git-update-index.

==Git Tips==
See [Development:Git tips] for git tips to help you with moodle development.

==Helpful Documentation==

* [http://www.kernel.org/pub/software/scm/git/docs/gitcvs-migration.html git for CVS users]
* [http://www.kernel.org/pub/software/scm/git/docs/ git man pages]
* [http://vger.kernel.org/vger-lists.html The Git mailing list]
* [http://git.or.cz/gitwiki GitWiki]
* [http://www.kernel.org/git/ kernel.org Gitweb interface] (Browse the Git and associated tools' repositories)
* [http://wiki.sourcemage.org/Git_Guide Straightforward Git guide] (for Git >= 1.5)

==Moodle Forum Discussions==

* [http://moodle.org/mod/forum/discuss.php?d=91998 Tracking Moodle CVS with git (take2)]
* [http://moodle.org/mod/forum/discuss.php?d=42275 Tracking Moodle CVS with git]
* [http://moodle.org/mod/forum/discuss.php?d=34472 Merging Custom Moodle Code With Stable Releases]
* [http://moodle.org/mod/forum/discuss.php?d=66412 Moving to git, need serious help]
* [http://moodle.org/mod/forum/discuss.php?d=99763 The continuing story of moving Moodle towards Git]

[[Category:Developer|Tracking Moodle CVS with git]]

Development:Tracking Moodle CVS with git

2008-10-02T11:10:07Z

Poltawski: /* Using git effectively with Moodle */

Note: Work in progress. These are notes for users that are comfortable with CVS and other SCMs, doing branching, merging, etc.

==Introduction==

If you plan on doing complicated or sustained development on Moodle, you will benefit from a local revision control system. Centralized systems like [http://www.nongnu.org/cvs/ CVS] and [http://subversion.tigris.org/ SVN] have limited capabilities for tracking vendor branches. For the ultimate experience you will want to use a system with distributed capabilities like [http://git.or.cz/ git]. Alternative tools for this include [http://www.selenic.com/mercurial/ Mercurial], [http://www.darcs.net/ Darcs] and [http://svk.elixus.org/ SVK].

GIT was developed by Linus Torvalds specifically for the Linux Kernel team. It is fast, fast, fast. Its usage is slightly different from CVS/SVN, but, if kernel developers can handle it ''Moodle developers will find it '''easy''' '' ;-)

Git is packaged for most major operating systems and an up to date list of packages can be found on [http://git.or.cz/ git.or.gz].

==Obtaining git==

You'll want:
* [http://git.or.cz/ Git] (and its deps). It includes '''gitk''', a very good UI for visualising project history.

You might also find the additional git porcelain (frontend) useful:
* [http://sourceforge.net/projects/qgit qgit] a nice GUI to view the project history, make commits, etc. Recommended! Note: qgit is tricky to compile by hand, get a .deb or an .rpm
* [http://www.procode.org/stgit/ stgit] (StackedGIT is mainly for users doing heavy cherry picking. Only recommended for very advanced SCM users.)

Git is still developing very quickly, but good mature versions are already available in packaged format. This guide is written with GIT v1.5.3 in mind.

== Downloading Moodle CVS History for git ==

There are currently two methods of retrieving Moodle CVS history for git:
* Clone the [http://www.catalyst.net.nz Catalyst IT] git import found at [http://git.catalyst.net.nz/gw?p=moodle-r2.git;a=summary git://git.catalyst.net.nz/moodle-r2.git]
* Run a local cvs import into git. ( Instructions for this can be found in [[Development:Importing Moodle CVS history into git]] )

It is '''highly advisable''' to clone the Catalyst git repository as this is updated hourly and the imported CVS history will share common commit hashes allowing developers to merge each others changes easily.

==Creating a Working Copy==

In order to use your repository you must clone yourself a working copy using [http://www.kernel.org/pub/software/scm/git/docs/git-clone.html git clone].

(Let's assume your destination directory is ~/src/moodle)

git clone git://git.catalyst.net.nz/moodle-r2.git ~/src/moodle

This will clone the catalyst moodle git repostitory into a local working copy that you may make changes to and commit to. This initial clone will take some time, but note that once you have cloned this repository, you will have the complete moodle source code history locally and can easily browse and checkout any commit in moodle history without a network connection.

==Using git effectively with Moodle==

Git can be used to intelligently track changes for creating client installs based off your own local, base, customisations which in turn tracks changes to the main Moodle CVS.

If you have a local git copy of the Moodle CVS repository you will have a number of heads such as: '''MOODLE_15_STABLE''', '''MOODLE_16_STABLE''', and so on. If you wish to create a new branch for your local customisations, you can base it off an existing HEAD (one of the CVS branches that exist in the main Moodle repository). If you wish to create a new local branch, '''mymoodle''', based off the current stable 1.6 code you would:

cd ~/src/moodle
git branch mymoodle origin/MOODLE_16_STABLE
git branch ''# lists all the current branches, should show '''mymoodle'''
git checkout mymoodle ''# you are now working on the '''mymoodle''' branch''
''# add some new files (i.e. themes, blocks, etc.)''
git add .
git status ''# Will list your uncommited files''
git commit -m "Added base customisations."

To return to the '''master''' HEAD, equivalent to the main Moodle CVS HEAD branch, you would have to use:

git branch master

To get updated from cvs/catalyst

cd ~/src/moodle
git fetch # fetches changes from upstream repostiroy
git checkout mymoodle
git merge origin/MOODLE_16_STABLE # merges changes from upstream

If you wanted to use this custom local branch to create a new client install, you could use:

git clone ~/src/moodle /home/someclient/public_html/
cd /home/someclient/public_html/
git checkout mymoodle origin/mymoodle

This would create a new git repository from your branch that could be used to do a client install which can, in turn, have the client's custom files added to it. This entire process can be automated so that your '''mymoodle''' branch is tracking upstream changes from the Moodle CVS ('''MOODLE_16_STABLE''' in this example) and your client installs are tracking changes against your local branch.

==Merging a patch from GIT into CVS==

If you have cvs access, you can use the [http://www.kernel.org/pub/software/scm/git/docs/git-cvsexportcommit.html git-cvsexportcommit] command to merge or cherry pick individual patches into ''upstream''.

::git-cvsexportcommit was initially written to do merges of NZOSVLE bugfixes into Moodle. Several tools and patches to GIT and Cogito come from the NZOSVLE team. [[User:Martin Langhoff|Martin Langhoff]] 14:55, 19 July 2006 (WST)

If the patches merge cleanly, future updates will recognise the already-applied patch. In that sense the merger in git is smarter than the merger in Cogito, consider using git-pull rather than cg-update. Note that if you see conflicts using git-pull you will have to work a bit harder to resolve them. Make sure you are familiar with handling merge conflicts with pure git before trying this.

==Preparing to merge large patch series or porting over to the next Moodle release==

When you have a large set of patches to apply to CVS, or a large set of custom patches to apply on top of the next release of Moodle, you can use [http://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html git-format-patch] to see what are your ''unmerged'' patches. git-format-patch tries hard to spot already-merged patches and skip them.

If you are applying patches to CVS, you can then use git-cvsexportcommit or plain old patch -p1 filename. If you are applying them to a new git branch (as you would to merge them on top of a new Moodle release) then you want to be on that new branch, and use git-am -3 -k filename.

If you are using git-am, make sure you learn how to deal with merge conflicts, and the use of git-update-index.

==Helpful Documentation==

* [http://www.kernel.org/pub/software/scm/git/docs/gitcvs-migration.html git for CVS users]
* [http://www.kernel.org/pub/software/scm/git/docs/ git man pages]
* [http://vger.kernel.org/vger-lists.html The Git mailing list]
* [http://git.or.cz/gitwiki GitWiki]
* [http://www.kernel.org/git/ kernel.org Gitweb interface] (Browse the Git and associated tools' repositories)
* [http://wiki.sourcemage.org/Git_Guide Straightforward Git guide] (for Git >= 1.5)

==Moodle Forum Discussions==

* [http://moodle.org/mod/forum/discuss.php?d=91998 Tracking Moodle CVS with git (take2)]
* [http://moodle.org/mod/forum/discuss.php?d=42275 Tracking Moodle CVS with git]
* [http://moodle.org/mod/forum/discuss.php?d=34472 Merging Custom Moodle Code With Stable Releases]
* [http://moodle.org/mod/forum/discuss.php?d=66412 Moving to git, need serious help]
* [http://moodle.org/mod/forum/discuss.php?d=99763 The continuing story of moving Moodle towards Git]

[[Category:Developer|Tracking Moodle CVS with git]]

Development:Tracking Moodle CVS with git

2008-10-02T11:06:09Z

Poltawski: /* Using git effectively with Moodle */

Note: Work in progress. These are notes for users that are comfortable with CVS and other SCMs, doing branching, merging, etc.

==Introduction==

If you plan on doing complicated or sustained development on Moodle, you will benefit from a local revision control system. Centralized systems like [http://www.nongnu.org/cvs/ CVS] and [http://subversion.tigris.org/ SVN] have limited capabilities for tracking vendor branches. For the ultimate experience you will want to use a system with distributed capabilities like [http://git.or.cz/ git]. Alternative tools for this include [http://www.selenic.com/mercurial/ Mercurial], [http://www.darcs.net/ Darcs] and [http://svk.elixus.org/ SVK].

GIT was developed by Linus Torvalds specifically for the Linux Kernel team. It is fast, fast, fast. Its usage is slightly different from CVS/SVN, but, if kernel developers can handle it ''Moodle developers will find it '''easy''' '' ;-)

Git is packaged for most major operating systems and an up to date list of packages can be found on [http://git.or.cz/ git.or.gz].

==Obtaining git==

You'll want:
* [http://git.or.cz/ Git] (and its deps). It includes '''gitk''', a very good UI for visualising project history.

You might also find the additional git porcelain (frontend) useful:
* [http://sourceforge.net/projects/qgit qgit] a nice GUI to view the project history, make commits, etc. Recommended! Note: qgit is tricky to compile by hand, get a .deb or an .rpm
* [http://www.procode.org/stgit/ stgit] (StackedGIT is mainly for users doing heavy cherry picking. Only recommended for very advanced SCM users.)

Git is still developing very quickly, but good mature versions are already available in packaged format. This guide is written with GIT v1.5.3 in mind.

== Downloading Moodle CVS History for git ==

There are currently two methods of retrieving Moodle CVS history for git:
* Clone the [http://www.catalyst.net.nz Catalyst IT] git import found at [http://git.catalyst.net.nz/gw?p=moodle-r2.git;a=summary git://git.catalyst.net.nz/moodle-r2.git]
* Run a local cvs import into git. ( Instructions for this can be found in [[Development:Importing Moodle CVS history into git]] )

It is '''highly advisable''' to clone the Catalyst git repository as this is updated hourly and the imported CVS history will share common commit hashes allowing developers to merge each others changes easily.

==Creating a Working Copy==

In order to use your repository you must clone yourself a working copy using [http://www.kernel.org/pub/software/scm/git/docs/git-clone.html git clone].

(Let's assume your destination directory is ~/src/moodle)

git clone git://git.catalyst.net.nz/moodle-r2.git ~/src/moodle

This will clone the catalyst moodle git repostitory into a local working copy that you may make changes to and commit to. This initial clone will take some time, but note that once you have cloned this repository, you will have the complete moodle source code history locally and can easily browse and checkout any commit in moodle history without a network connection.

==Using git effectively with Moodle==

Git can be used to intelligently track changes for creating client installs based off your own local, base, customisations which in turn tracks changes to the main Moodle CVS.

If you have a local git copy of the Moodle CVS repository you will have a number of heads such as: '''MOODLE_15_STABLE''', '''MOODLE_16_STABLE''', and so on. If you wish to create a new branch for your local customisations, you can base it off an existing HEAD (one of the CVS branches that exist in the main Moodle repository). If you wish to create a new local branch, '''mymoodle''', based off the current stable 1.6 code you would:

cd ~/src/moodle
git branch mymoodle origin/MOODLE_16_STABLE
git branch ''# lists all the current branches, should show '''mymoodle'''
git checkout mymoodle ''# you are now working on the '''mymoodle''' branch''
''# add some new files (i.e. themes, blocks, etc.)''
git add .
git status ''# Will list your uncommited files''
git commit -m "Added base customisations."

To return to the '''master''' HEAD, equivalent to the main Moodle CVS HEAD branch, you would have to use:

git branch master

If you were running a nightly script to pull new Moodle CVS updates into your main git repository via git-cvsimport, you should also merge any new changes into your local branch:

cd ~/src/moodle.git
git checkout mymoodle
git merge origin/MOODLE_16_STABLE

If you wanted to use this custom local branch to create a new client install, you could use:

git clone ~/src/moodle /home/someclient/public_html/
cd /home/someclient/public_html/
git checkout mymoodle origin/mymoodle

This would create a new git repository from your branch that could be used to do a client install which can, in turn, have the client's custom files added to it. This entire process can be automated so that your '''mymoodle''' branch is tracking upstream changes from the Moodle CVS ('''MOODLE_16_STABLE''' in this example) and your client installs are tracking changes against your local branch.

==Merging a patch from GIT into CVS==

If you have cvs access, you can use the [http://www.kernel.org/pub/software/scm/git/docs/git-cvsexportcommit.html git-cvsexportcommit] command to merge or cherry pick individual patches into ''upstream''.

::git-cvsexportcommit was initially written to do merges of NZOSVLE bugfixes into Moodle. Several tools and patches to GIT and Cogito come from the NZOSVLE team. [[User:Martin Langhoff|Martin Langhoff]] 14:55, 19 July 2006 (WST)

If the patches merge cleanly, future updates will recognise the already-applied patch. In that sense the merger in git is smarter than the merger in Cogito, consider using git-pull rather than cg-update. Note that if you see conflicts using git-pull you will have to work a bit harder to resolve them. Make sure you are familiar with handling merge conflicts with pure git before trying this.

==Preparing to merge large patch series or porting over to the next Moodle release==

When you have a large set of patches to apply to CVS, or a large set of custom patches to apply on top of the next release of Moodle, you can use [http://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html git-format-patch] to see what are your ''unmerged'' patches. git-format-patch tries hard to spot already-merged patches and skip them.

If you are applying patches to CVS, you can then use git-cvsexportcommit or plain old patch -p1 filename. If you are applying them to a new git branch (as you would to merge them on top of a new Moodle release) then you want to be on that new branch, and use git-am -3 -k filename.

If you are using git-am, make sure you learn how to deal with merge conflicts, and the use of git-update-index.

==Helpful Documentation==

* [http://www.kernel.org/pub/software/scm/git/docs/gitcvs-migration.html git for CVS users]
* [http://www.kernel.org/pub/software/scm/git/docs/ git man pages]
* [http://vger.kernel.org/vger-lists.html The Git mailing list]
* [http://git.or.cz/gitwiki GitWiki]
* [http://www.kernel.org/git/ kernel.org Gitweb interface] (Browse the Git and associated tools' repositories)
* [http://wiki.sourcemage.org/Git_Guide Straightforward Git guide] (for Git >= 1.5)

==Moodle Forum Discussions==

* [http://moodle.org/mod/forum/discuss.php?d=91998 Tracking Moodle CVS with git (take2)]
* [http://moodle.org/mod/forum/discuss.php?d=42275 Tracking Moodle CVS with git]
* [http://moodle.org/mod/forum/discuss.php?d=34472 Merging Custom Moodle Code With Stable Releases]
* [http://moodle.org/mod/forum/discuss.php?d=66412 Moving to git, need serious help]
* [http://moodle.org/mod/forum/discuss.php?d=99763 The continuing story of moving Moodle towards Git]

[[Category:Developer|Tracking Moodle CVS with git]]

Development:Tracking Moodle CVS with git

2008-10-02T11:00:05Z

Poltawski: added info about the time of initial clone

Note: Work in progress. These are notes for users that are comfortable with CVS and other SCMs, doing branching, merging, etc.

==Introduction==

If you plan on doing complicated or sustained development on Moodle, you will benefit from a local revision control system. Centralized systems like [http://www.nongnu.org/cvs/ CVS] and [http://subversion.tigris.org/ SVN] have limited capabilities for tracking vendor branches. For the ultimate experience you will want to use a system with distributed capabilities like [http://git.or.cz/ git]. Alternative tools for this include [http://www.selenic.com/mercurial/ Mercurial], [http://www.darcs.net/ Darcs] and [http://svk.elixus.org/ SVK].

GIT was developed by Linus Torvalds specifically for the Linux Kernel team. It is fast, fast, fast. Its usage is slightly different from CVS/SVN, but, if kernel developers can handle it ''Moodle developers will find it '''easy''' '' ;-)

Git is packaged for most major operating systems and an up to date list of packages can be found on [http://git.or.cz/ git.or.gz].

==Obtaining git==

You'll want:
* [http://git.or.cz/ Git] (and its deps). It includes '''gitk''', a very good UI for visualising project history.

You might also find the additional git porcelain (frontend) useful:
* [http://sourceforge.net/projects/qgit qgit] a nice GUI to view the project history, make commits, etc. Recommended! Note: qgit is tricky to compile by hand, get a .deb or an .rpm
* [http://www.procode.org/stgit/ stgit] (StackedGIT is mainly for users doing heavy cherry picking. Only recommended for very advanced SCM users.)

Git is still developing very quickly, but good mature versions are already available in packaged format. This guide is written with GIT v1.5.3 in mind.

== Downloading Moodle CVS History for git ==

There are currently two methods of retrieving Moodle CVS history for git:
* Clone the [http://www.catalyst.net.nz Catalyst IT] git import found at [http://git.catalyst.net.nz/gw?p=moodle-r2.git;a=summary git://git.catalyst.net.nz/moodle-r2.git]
* Run a local cvs import into git. ( Instructions for this can be found in [[Development:Importing Moodle CVS history into git]] )

It is '''highly advisable''' to clone the Catalyst git repository as this is updated hourly and the imported CVS history will share common commit hashes allowing developers to merge each others changes easily.

==Creating a Working Copy==

In order to use your repository you must clone yourself a working copy using [http://www.kernel.org/pub/software/scm/git/docs/git-clone.html git clone].

(Let's assume your destination directory is ~/src/moodle)

git clone git://git.catalyst.net.nz/moodle-r2.git ~/src/moodle

This will clone the catalyst moodle git repostitory into a local working copy that you may make changes to and commit to. This initial clone will take some time, but note that once you have cloned this repository, you will have the complete moodle source code history locally and can easily browse and checkout any commit in moodle history without a network connection.

==Using git effectively with Moodle==

Git can be used to intelligently track changes for creating client installs based off your own local, base, customisations which in turn tracks changes to the main Moodle CVS.

If you have a local git copy of the Moodle CVS repository you will have a number of heads such as: '''MOODLE_15_STABLE''', '''MOODLE_16_STABLE''', and so on. If you wish to create a new branch for your local customisations, you can base it off an existing HEAD (one of the CVS branches that exist in the main Moodle repository). If you wish to create a new local branch, '''mymoodle''', based off the current stable 1.6 code you would:

cd ~/src/moodle
git branch mymoodle origin/MOODLE_16_STABLE
git branch ''# lists all the current branches, should show '''mymoodle'''
git checkout mymoodle ''# you are now working on the '''mymoodle''' branch''
''# add some new files (i.e. themes, blocks, etc.)''
git add .
git status ''# Will list your uncommited files''
git commit -m "Added base customisations."

To return to the '''master''' HEAD, equivalent to the main Moodle CVS HEAD branch, you would have to use:

git branch master

If you were running a nightly script to pull new Moodle CVS updates into your main git repository via git-cvsimport, you should also merge any new changes into your local branch:

cd ~/src/moodle.git
git checkout mymoodle
git merge origin/MOODLE_16_STABLE

If you wanted to use this custom local branch to create a new client install, you could use:

git clone ~src/moodle /home/someclient/public_html/
cd /home/someclient/public_html/
git checkout mymoodle origin/mymoodle

This would create a new git repository from your branch that could be used to do a client install which can, in turn, have the client's custom files added to it. This entire process can be automated so that your '''mymoodle''' branch is tracking upstream changes from the Moodle CVS ('''MOODLE_16_STABLE''' in this example) and your client installs are tracking changes against your local branch.

==Merging a patch from GIT into CVS==

If you have cvs access, you can use the [http://www.kernel.org/pub/software/scm/git/docs/git-cvsexportcommit.html git-cvsexportcommit] command to merge or cherry pick individual patches into ''upstream''.

::git-cvsexportcommit was initially written to do merges of NZOSVLE bugfixes into Moodle. Several tools and patches to GIT and Cogito come from the NZOSVLE team. [[User:Martin Langhoff|Martin Langhoff]] 14:55, 19 July 2006 (WST)

If the patches merge cleanly, future updates will recognise the already-applied patch. In that sense the merger in git is smarter than the merger in Cogito, consider using git-pull rather than cg-update. Note that if you see conflicts using git-pull you will have to work a bit harder to resolve them. Make sure you are familiar with handling merge conflicts with pure git before trying this.

==Preparing to merge large patch series or porting over to the next Moodle release==

When you have a large set of patches to apply to CVS, or a large set of custom patches to apply on top of the next release of Moodle, you can use [http://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html git-format-patch] to see what are your ''unmerged'' patches. git-format-patch tries hard to spot already-merged patches and skip them.

If you are applying patches to CVS, you can then use git-cvsexportcommit or plain old patch -p1 filename. If you are applying them to a new git branch (as you would to merge them on top of a new Moodle release) then you want to be on that new branch, and use git-am -3 -k filename.

If you are using git-am, make sure you learn how to deal with merge conflicts, and the use of git-update-index.

==Helpful Documentation==

* [http://www.kernel.org/pub/software/scm/git/docs/gitcvs-migration.html git for CVS users]
* [http://www.kernel.org/pub/software/scm/git/docs/ git man pages]
* [http://vger.kernel.org/vger-lists.html The Git mailing list]
* [http://git.or.cz/gitwiki GitWiki]
* [http://www.kernel.org/git/ kernel.org Gitweb interface] (Browse the Git and associated tools' repositories)
* [http://wiki.sourcemage.org/Git_Guide Straightforward Git guide] (for Git >= 1.5)

==Moodle Forum Discussions==

* [http://moodle.org/mod/forum/discuss.php?d=91998 Tracking Moodle CVS with git (take2)]
* [http://moodle.org/mod/forum/discuss.php?d=42275 Tracking Moodle CVS with git]
* [http://moodle.org/mod/forum/discuss.php?d=34472 Merging Custom Moodle Code With Stable Releases]
* [http://moodle.org/mod/forum/discuss.php?d=66412 Moving to git, need serious help]
* [http://moodle.org/mod/forum/discuss.php?d=99763 The continuing story of moving Moodle towards Git]

[[Category:Developer|Tracking Moodle CVS with git]]

Development:Tracking Moodle CVS with git

2008-10-02T10:51:20Z

Poltawski: /* Obtaining git */

Note: Work in progress. These are notes for users that are comfortable with CVS and other SCMs, doing branching, merging, etc.

==Introduction==

If you plan on doing complicated or sustained development on Moodle, you will benefit from a local revision control system. Centralized systems like [http://www.nongnu.org/cvs/ CVS] and [http://subversion.tigris.org/ SVN] have limited capabilities for tracking vendor branches. For the ultimate experience you will want to use a system with distributed capabilities like [http://git.or.cz/ git]. Alternative tools for this include [http://www.selenic.com/mercurial/ Mercurial], [http://www.darcs.net/ Darcs] and [http://svk.elixus.org/ SVK].

GIT was developed by Linus Torvalds specifically for the Linux Kernel team. It is fast, fast, fast. Its usage is slightly different from CVS/SVN, but, if kernel developers can handle it ''Moodle developers will find it '''easy''' '' ;-)

Git is packaged for most major operating systems and an up to date list of packages can be found on [http://git.or.cz/ git.or.gz].

==Obtaining git==

You'll want:
* [http://git.or.cz/ Git] (and its deps). It includes '''gitk''', a very good UI for visualising project history.

You might also find the additional git porcelain (frontend) useful:
* [http://sourceforge.net/projects/qgit qgit] a nice GUI to view the project history, make commits, etc. Recommended! Note: qgit is tricky to compile by hand, get a .deb or an .rpm
* [http://www.procode.org/stgit/ stgit] (StackedGIT is mainly for users doing heavy cherry picking. Only recommended for very advanced SCM users.)

Git is still developing very quickly, but good mature versions are already available in packaged format. This guide is written with GIT v1.5.3 in mind.

== Downloading Moodle CVS History for git ==

There are currently two methods of retrieving Moodle CVS history for git:
* Clone the [http://www.catalyst.net.nz Catalyst IT] git import found at [http://git.catalyst.net.nz/gw?p=moodle-r2.git;a=summary git://git.catalyst.net.nz/moodle-r2.git]
* Run a local cvs import into git. ( Instructions for this can be found in [[Development:Importing Moodle CVS history into git]] )

It is '''highly advisable''' to clone the Catalyst git repository as this is updated hourly and the imported CVS history will share common commit hashes allowing developers to merge each others changes easily.

==Creating a Working Copy==

In order to use your repository you must clone yourself a working copy using [http://www.kernel.org/pub/software/scm/git/docs/git-clone.html git clone].

(Let's assume your destination directory is ~/src/moodle)

git clone git://git.catalyst.net.nz/moodle-r2.git ~/src/moodle

This will clone the catalyst moodle git repostitory into a local working copy that you may make changes to and commit to. You can then get updates from the catalyst git repository that can be merged in amongst the changes to your local copy.

==Using git effectively with Moodle==

Git can be used to intelligently track changes for creating client installs based off your own local, base, customisations which in turn tracks changes to the main Moodle CVS.

If you have a local git copy of the Moodle CVS repository you will have a number of heads such as: '''MOODLE_15_STABLE''', '''MOODLE_16_STABLE''', and so on. If you wish to create a new branch for your local customisations, you can base it off an existing HEAD (one of the CVS branches that exist in the main Moodle repository). If you wish to create a new local branch, '''mymoodle''', based off the current stable 1.6 code you would:

cd ~/src/moodle
git branch mymoodle origin/MOODLE_16_STABLE
git branch ''# lists all the current branches, should show '''mymoodle'''
git checkout mymoodle ''# you are now working on the '''mymoodle''' branch''
''# add some new files (i.e. themes, blocks, etc.)''
git add .
git status ''# Will list your uncommited files''
git commit -m "Added base customisations."

To return to the '''master''' HEAD, equivalent to the main Moodle CVS HEAD branch, you would have to use:

git branch master

If you were running a nightly script to pull new Moodle CVS updates into your main git repository via git-cvsimport, you should also merge any new changes into your local branch:

cd ~/src/moodle.git
git checkout mymoodle
git merge origin/MOODLE_16_STABLE

If you wanted to use this custom local branch to create a new client install, you could use:

git clone ~src/moodle /home/someclient/public_html/
cd /home/someclient/public_html/
git checkout mymoodle origin/mymoodle

This would create a new git repository from your branch that could be used to do a client install which can, in turn, have the client's custom files added to it. This entire process can be automated so that your '''mymoodle''' branch is tracking upstream changes from the Moodle CVS ('''MOODLE_16_STABLE''' in this example) and your client installs are tracking changes against your local branch.

==Merging a patch from GIT into CVS==

If you have cvs access, you can use the [http://www.kernel.org/pub/software/scm/git/docs/git-cvsexportcommit.html git-cvsexportcommit] command to merge or cherry pick individual patches into ''upstream''.

::git-cvsexportcommit was initially written to do merges of NZOSVLE bugfixes into Moodle. Several tools and patches to GIT and Cogito come from the NZOSVLE team. [[User:Martin Langhoff|Martin Langhoff]] 14:55, 19 July 2006 (WST)

If the patches merge cleanly, future updates will recognise the already-applied patch. In that sense the merger in git is smarter than the merger in Cogito, consider using git-pull rather than cg-update. Note that if you see conflicts using git-pull you will have to work a bit harder to resolve them. Make sure you are familiar with handling merge conflicts with pure git before trying this.

==Preparing to merge large patch series or porting over to the next Moodle release==

When you have a large set of patches to apply to CVS, or a large set of custom patches to apply on top of the next release of Moodle, you can use [http://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html git-format-patch] to see what are your ''unmerged'' patches. git-format-patch tries hard to spot already-merged patches and skip them.

If you are applying patches to CVS, you can then use git-cvsexportcommit or plain old patch -p1 filename. If you are applying them to a new git branch (as you would to merge them on top of a new Moodle release) then you want to be on that new branch, and use git-am -3 -k filename.

If you are using git-am, make sure you learn how to deal with merge conflicts, and the use of git-update-index.

==Helpful Documentation==

* [http://www.kernel.org/pub/software/scm/git/docs/gitcvs-migration.html git for CVS users]
* [http://www.kernel.org/pub/software/scm/git/docs/ git man pages]
* [http://vger.kernel.org/vger-lists.html The Git mailing list]
* [http://git.or.cz/gitwiki GitWiki]
* [http://www.kernel.org/git/ kernel.org Gitweb interface] (Browse the Git and associated tools' repositories)
* [http://wiki.sourcemage.org/Git_Guide Straightforward Git guide] (for Git >= 1.5)

==Moodle Forum Discussions==

* [http://moodle.org/mod/forum/discuss.php?d=91998 Tracking Moodle CVS with git (take2)]
* [http://moodle.org/mod/forum/discuss.php?d=42275 Tracking Moodle CVS with git]
* [http://moodle.org/mod/forum/discuss.php?d=34472 Merging Custom Moodle Code With Stable Releases]
* [http://moodle.org/mod/forum/discuss.php?d=66412 Moving to git, need serious help]
* [http://moodle.org/mod/forum/discuss.php?d=99763 The continuing story of moving Moodle towards Git]

[[Category:Developer|Tracking Moodle CVS with git]]

Development:Tracking Moodle CVS with git

2008-10-02T10:48:35Z

Poltawski: /* Using git effectively with Moodle */

Note: Work in progress. These are notes for users that are comfortable with CVS and other SCMs, doing branching, merging, etc.

==Introduction==

If you plan on doing complicated or sustained development on Moodle, you will benefit from a local revision control system. Centralized systems like [http://www.nongnu.org/cvs/ CVS] and [http://subversion.tigris.org/ SVN] have limited capabilities for tracking vendor branches. For the ultimate experience you will want to use a system with distributed capabilities like [http://git.or.cz/ git]. Alternative tools for this include [http://www.selenic.com/mercurial/ Mercurial], [http://www.darcs.net/ Darcs] and [http://svk.elixus.org/ SVK].

GIT was developed by Linus Torvalds specifically for the Linux Kernel team. It is fast, fast, fast. Its usage is slightly different from CVS/SVN, but, if kernel developers can handle it ''Moodle developers will find it '''easy''' '' ;-)

Git is packaged for most major operating systems and an up to date list of packages can be found on [http://git.or.cz/ git.or.gz].

==Obtaining git==

You'll want:
* [http://www.kernel.org/pub/software/scm/git/ Git] (and its deps). It includes '''gitk''', a very good UI for visualising project history.
* an additional git porcelain (frontend)
** [http://sourceforge.net/projects/qgit qgit] a nice GUI to view the project history, make commits, etc. Recommended! Note: qgit is tricky to compile by hand, get a .deb or an .rpm
** [http://www.procode.org/stgit/ stgit] (StackedGIT is mainly for users doing heavy cherry picking. Only recommended for very advanced SCM users.)

Git is still developing quickly, but good mature versions are already available in packaged format. This guide is written with GIT v1.5.3 in mind.

== Downloading Moodle CVS History for git ==

There are currently two methods of retrieving Moodle CVS history for git:
* Clone the [http://www.catalyst.net.nz Catalyst IT] git import found at [http://git.catalyst.net.nz/gw?p=moodle-r2.git;a=summary git://git.catalyst.net.nz/moodle-r2.git]
* Run a local cvs import into git. ( Instructions for this can be found in [[Development:Importing Moodle CVS history into git]] )

It is '''highly advisable''' to clone the Catalyst git repository as this is updated hourly and the imported CVS history will share common commit hashes allowing developers to merge each others changes easily.

==Creating a Working Copy==

In order to use your repository you must clone yourself a working copy using [http://www.kernel.org/pub/software/scm/git/docs/git-clone.html git clone].

(Let's assume your destination directory is ~/src/moodle)

git clone git://git.catalyst.net.nz/moodle-r2.git ~/src/moodle

This will clone the catalyst moodle git repostitory into a local working copy that you may make changes to and commit to. You can then get updates from the catalyst git repository that can be merged in amongst the changes to your local copy.

==Using git effectively with Moodle==

Git can be used to intelligently track changes for creating client installs based off your own local, base, customisations which in turn tracks changes to the main Moodle CVS.

If you have a local git copy of the Moodle CVS repository you will have a number of heads such as: '''MOODLE_15_STABLE''', '''MOODLE_16_STABLE''', and so on. If you wish to create a new branch for your local customisations, you can base it off an existing HEAD (one of the CVS branches that exist in the main Moodle repository). If you wish to create a new local branch, '''mymoodle''', based off the current stable 1.6 code you would:

cd ~/src/moodle
git branch mymoodle origin/MOODLE_16_STABLE
git branch ''# lists all the current branches, should show '''mymoodle'''
git checkout mymoodle ''# you are now working on the '''mymoodle''' branch''
''# add some new files (i.e. themes, blocks, etc.)''
git add .
git status ''# Will list your uncommited files''
git commit -m "Added base customisations."

To return to the '''master''' HEAD, equivalent to the main Moodle CVS HEAD branch, you would have to use:

git branch master

If you were running a nightly script to pull new Moodle CVS updates into your main git repository via git-cvsimport, you should also merge any new changes into your local branch:

cd ~/src/moodle.git
git checkout mymoodle
git merge origin/MOODLE_16_STABLE

If you wanted to use this custom local branch to create a new client install, you could use:

git clone ~src/moodle /home/someclient/public_html/
cd /home/someclient/public_html/
git checkout mymoodle origin/mymoodle

This would create a new git repository from your branch that could be used to do a client install which can, in turn, have the client's custom files added to it. This entire process can be automated so that your '''mymoodle''' branch is tracking upstream changes from the Moodle CVS ('''MOODLE_16_STABLE''' in this example) and your client installs are tracking changes against your local branch.

==Merging a patch from GIT into CVS==

If you have cvs access, you can use the [http://www.kernel.org/pub/software/scm/git/docs/git-cvsexportcommit.html git-cvsexportcommit] command to merge or cherry pick individual patches into ''upstream''.

::git-cvsexportcommit was initially written to do merges of NZOSVLE bugfixes into Moodle. Several tools and patches to GIT and Cogito come from the NZOSVLE team. [[User:Martin Langhoff|Martin Langhoff]] 14:55, 19 July 2006 (WST)

If the patches merge cleanly, future updates will recognise the already-applied patch. In that sense the merger in git is smarter than the merger in Cogito, consider using git-pull rather than cg-update. Note that if you see conflicts using git-pull you will have to work a bit harder to resolve them. Make sure you are familiar with handling merge conflicts with pure git before trying this.

==Preparing to merge large patch series or porting over to the next Moodle release==

When you have a large set of patches to apply to CVS, or a large set of custom patches to apply on top of the next release of Moodle, you can use [http://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html git-format-patch] to see what are your ''unmerged'' patches. git-format-patch tries hard to spot already-merged patches and skip them.

If you are applying patches to CVS, you can then use git-cvsexportcommit or plain old patch -p1 filename. If you are applying them to a new git branch (as you would to merge them on top of a new Moodle release) then you want to be on that new branch, and use git-am -3 -k filename.

If you are using git-am, make sure you learn how to deal with merge conflicts, and the use of git-update-index.

==Helpful Documentation==

* [http://www.kernel.org/pub/software/scm/git/docs/gitcvs-migration.html git for CVS users]
* [http://www.kernel.org/pub/software/scm/git/docs/ git man pages]
* [http://vger.kernel.org/vger-lists.html The Git mailing list]
* [http://git.or.cz/gitwiki GitWiki]
* [http://www.kernel.org/git/ kernel.org Gitweb interface] (Browse the Git and associated tools' repositories)
* [http://wiki.sourcemage.org/Git_Guide Straightforward Git guide] (for Git >= 1.5)

==Moodle Forum Discussions==

* [http://moodle.org/mod/forum/discuss.php?d=91998 Tracking Moodle CVS with git (take2)]
* [http://moodle.org/mod/forum/discuss.php?d=42275 Tracking Moodle CVS with git]
* [http://moodle.org/mod/forum/discuss.php?d=34472 Merging Custom Moodle Code With Stable Releases]
* [http://moodle.org/mod/forum/discuss.php?d=66412 Moving to git, need serious help]
* [http://moodle.org/mod/forum/discuss.php?d=99763 The continuing story of moving Moodle towards Git]

[[Category:Developer|Tracking Moodle CVS with git]]

Development:Tracking Moodle CVS with git

2008-10-02T10:46:21Z

Poltawski: /* Helpful Documentation */

Note: Work in progress. These are notes for users that are comfortable with CVS and other SCMs, doing branching, merging, etc.

==Introduction==

If you plan on doing complicated or sustained development on Moodle, you will benefit from a local revision control system. Centralized systems like [http://www.nongnu.org/cvs/ CVS] and [http://subversion.tigris.org/ SVN] have limited capabilities for tracking vendor branches. For the ultimate experience you will want to use a system with distributed capabilities like [http://git.or.cz/ git]. Alternative tools for this include [http://www.selenic.com/mercurial/ Mercurial], [http://www.darcs.net/ Darcs] and [http://svk.elixus.org/ SVK].

GIT was developed by Linus Torvalds specifically for the Linux Kernel team. It is fast, fast, fast. Its usage is slightly different from CVS/SVN, but, if kernel developers can handle it ''Moodle developers will find it '''easy''' '' ;-)

Git is packaged for most major operating systems and an up to date list of packages can be found on [http://git.or.cz/ git.or.gz].

==Obtaining git==

You'll want:
* [http://www.kernel.org/pub/software/scm/git/ Git] (and its deps). It includes '''gitk''', a very good UI for visualising project history.
* an additional git porcelain (frontend)
** [http://sourceforge.net/projects/qgit qgit] a nice GUI to view the project history, make commits, etc. Recommended! Note: qgit is tricky to compile by hand, get a .deb or an .rpm
** [http://www.procode.org/stgit/ stgit] (StackedGIT is mainly for users doing heavy cherry picking. Only recommended for very advanced SCM users.)

Git is still developing quickly, but good mature versions are already available in packaged format. This guide is written with GIT v1.5.3 in mind.

== Downloading Moodle CVS History for git ==

There are currently two methods of retrieving Moodle CVS history for git:
* Clone the [http://www.catalyst.net.nz Catalyst IT] git import found at [http://git.catalyst.net.nz/gw?p=moodle-r2.git;a=summary git://git.catalyst.net.nz/moodle-r2.git]
* Run a local cvs import into git. ( Instructions for this can be found in [[Development:Importing Moodle CVS history into git]] )

It is '''highly advisable''' to clone the Catalyst git repository as this is updated hourly and the imported CVS history will share common commit hashes allowing developers to merge each others changes easily.

==Creating a Working Copy==

In order to use your repository you must clone yourself a working copy using [http://www.kernel.org/pub/software/scm/git/docs/git-clone.html git clone].

(Let's assume your destination directory is ~/src/moodle)

git clone git://git.catalyst.net.nz/moodle-r2.git ~/src/moodle

This will clone the catalyst moodle git repostitory into a local working copy that you may make changes to and commit to. You can then get updates from the catalyst git repository that can be merged in amongst the changes to your local copy.

==Using git effectively with Moodle==

Git can be used to intelligently track changes for creating client installs based off your own local, base, customisations which in turn tracks changes to the main Moodle CVS.

If you have a local git copy of the Moodle CVS repository you will have a number of heads such as: '''MOODLE_15_STABLE''', '''MOODLE_16_STABLE''', and so on. If you wish to create a new branch for your local customisations, you can base it off an existing HEAD (one of the CVS branches that exist in the main Moodle repository). If you wish to create a new local branch, '''mymoodle''', based off the current stable 1.6 code you would:

cd ~/src/moodle
git branch mymoodle MOODLE_16_STABLE
git branch ''# lists all the current branches, should show '''mymoodle'''
git checkout mymoodle ''# you are now working on the '''mymoodle''' branch''
''# add some new files (i.e. themes, blocks, etc.)''
git add .
git status ''# Will list your uncommited files''
git commit -m "Added base customisations."

To return to the '''master''' HEAD, equivalent to the main Moodle CVS HEAD branch, you would have to use:

git branch master

If you were running a nightly script to pull new Moodle CVS updates into your main git repository via git-cvsimport, you should also merge any new changes into your local branch:

cd ~/src/moodle.git
git checkout mymoodle
git merge MOODLE_16_STABLE

If you wanted to use this custom local branch to create a new client install, you could use:

git-clone ~src/moodle /home/someclient/public_html/
cd /home/someclient/public_html/
git checkout mymoodle origin/mymoodle

This would create a new git repository from your branch that could be used to do a client install which can, in turn, have the client's custom files added to it. This entire process can be automated so that your '''mymoodle''' branch is tracking upstream changes from the Moodle CVS ('''MOODLE_16_STABLE''' in this example) and your client installs are tracking changes against your local branch.

==Merging a patch from GIT into CVS==

If you have cvs access, you can use the [http://www.kernel.org/pub/software/scm/git/docs/git-cvsexportcommit.html git-cvsexportcommit] command to merge or cherry pick individual patches into ''upstream''.

::git-cvsexportcommit was initially written to do merges of NZOSVLE bugfixes into Moodle. Several tools and patches to GIT and Cogito come from the NZOSVLE team. [[User:Martin Langhoff|Martin Langhoff]] 14:55, 19 July 2006 (WST)

If the patches merge cleanly, future updates will recognise the already-applied patch. In that sense the merger in git is smarter than the merger in Cogito, consider using git-pull rather than cg-update. Note that if you see conflicts using git-pull you will have to work a bit harder to resolve them. Make sure you are familiar with handling merge conflicts with pure git before trying this.

==Preparing to merge large patch series or porting over to the next Moodle release==

When you have a large set of patches to apply to CVS, or a large set of custom patches to apply on top of the next release of Moodle, you can use [http://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html git-format-patch] to see what are your ''unmerged'' patches. git-format-patch tries hard to spot already-merged patches and skip them.

If you are applying patches to CVS, you can then use git-cvsexportcommit or plain old patch -p1 filename. If you are applying them to a new git branch (as you would to merge them on top of a new Moodle release) then you want to be on that new branch, and use git-am -3 -k filename.

If you are using git-am, make sure you learn how to deal with merge conflicts, and the use of git-update-index.

==Helpful Documentation==

* [http://www.kernel.org/pub/software/scm/git/docs/gitcvs-migration.html git for CVS users]
* [http://www.kernel.org/pub/software/scm/git/docs/ git man pages]
* [http://vger.kernel.org/vger-lists.html The Git mailing list]
* [http://git.or.cz/gitwiki GitWiki]
* [http://www.kernel.org/git/ kernel.org Gitweb interface] (Browse the Git and associated tools' repositories)
* [http://wiki.sourcemage.org/Git_Guide Straightforward Git guide] (for Git >= 1.5)

==Moodle Forum Discussions==

* [http://moodle.org/mod/forum/discuss.php?d=91998 Tracking Moodle CVS with git (take2)]
* [http://moodle.org/mod/forum/discuss.php?d=42275 Tracking Moodle CVS with git]
* [http://moodle.org/mod/forum/discuss.php?d=34472 Merging Custom Moodle Code With Stable Releases]
* [http://moodle.org/mod/forum/discuss.php?d=66412 Moving to git, need serious help]
* [http://moodle.org/mod/forum/discuss.php?d=99763 The continuing story of moving Moodle towards Git]

[[Category:Developer|Tracking Moodle CVS with git]]

Development:Importing Moodle CVS history into git

2008-10-02T10:46:13Z

Poltawski:

==Importing CVS==

To begin we must import the CVS repository into a local git filesystem using [http://www.kernel.org/pub/software/scm/git/docs/git-cvsimport.html git-cvsimport].

#!/bin/bash
CVSROOT=:pserver:anonymous@uk.cvs.moodle.org:/cvsroot/moodle
MODULE=moodle
INSTALLDIR=moodle
git-cvsimport -p x -v -k -o cvshead -d $CVSROOT -C $INSTALLDIR $MODULE &> cvsimport.log

(When might one want to use the -m -u and -s flags?)
-u purely cosmetic
-s slashes can cause problems but this doesn't seem to be the case with moodle
-m ?

This will run for a '''very''' long time as it checks out every revision of each file (including all branches). However, if you run the command after a successful initial run it will simply get the new revisions since it was last run. This creates a new git repository which can then be cloned and worked upon.

At the time of writing the size of a new Moodle git repository was approximately 866MB ''unpacked'', and 75MB packed. Newer versions of git pack the project during import, so your resulting repository plus checkout should be a bit over 100MB. Still, it is a good idea to pack the repository to make it smaller and faster, running

git-repack -a -d

With the -o cvshead flag, the "HEAD" of the CVS repo will appear as 'cvshead' in GIT, making things a lot clearer.

==Importing CVS Faster==

For initial imports, it is highly recommended that you fetch the CVS repository from one of Moodle [[CVS_for_Administrators#CVS_Servers | cvs servers]] and run the initial cvsimport against your local copy of the repository.

:: For the more adventurous: Keith Packard wrote another importer that does a better job at the initial import, called parsecvs. I use parsecvs for the initial import and then git-cvsimport daily for the incremental imports. --[[User:Martin Langhoff|Martin Langhoff]] 15:04, 19 July 2006 (WST)

:: It took me a few minutes to find the info in the Sourceforge docs on how to fetch the raw CVS repo via rsync. Just type 'rsync -av "rsync://moodle.cvs.sourceforge.net/cvsroot/moodle/moodle/*" .' (without the single quotes) and you'll be done --[[User:Iñaki Arenaza|Iñaki Arenaza]] 17:54, 25 December 2006 (CST).

:: This rsync command line doesn't work as-is anymore, since the repository was moved from sourceforge. Some mirrors (at least es.cvs.moodle.org at this time) provide rsync, but some don't (like eu.cvs.moodle.org). Find the best mirror for you and update the rsync call accordingly.

After the initial import, you can run git-cvsimport against your favorite cvs.moodle.org mirror repository, or update your local copy via rsync as sas demonstrated in [http://www.progsoc.org/~wildfire/git/update-repo.sh this script].

== Useful Links ==

* [http://www.kernel.org/pub/software/scm/git/docs/git-cvsimport.html git-cvsimport man page]

[[Category:Developer|Importing Moodle CVS history into git]]

Development:Tracking Moodle CVS with git

2008-10-02T10:45:21Z

Poltawski: /* Creating a Working Copy */

Note: Work in progress. These are notes for users that are comfortable with CVS and other SCMs, doing branching, merging, etc.

==Introduction==

If you plan on doing complicated or sustained development on Moodle, you will benefit from a local revision control system. Centralized systems like [http://www.nongnu.org/cvs/ CVS] and [http://subversion.tigris.org/ SVN] have limited capabilities for tracking vendor branches. For the ultimate experience you will want to use a system with distributed capabilities like [http://git.or.cz/ git]. Alternative tools for this include [http://www.selenic.com/mercurial/ Mercurial], [http://www.darcs.net/ Darcs] and [http://svk.elixus.org/ SVK].

GIT was developed by Linus Torvalds specifically for the Linux Kernel team. It is fast, fast, fast. Its usage is slightly different from CVS/SVN, but, if kernel developers can handle it ''Moodle developers will find it '''easy''' '' ;-)

Git is packaged for most major operating systems and an up to date list of packages can be found on [http://git.or.cz/ git.or.gz].

==Obtaining git==

You'll want:
* [http://www.kernel.org/pub/software/scm/git/ Git] (and its deps). It includes '''gitk''', a very good UI for visualising project history.
* an additional git porcelain (frontend)
** [http://sourceforge.net/projects/qgit qgit] a nice GUI to view the project history, make commits, etc. Recommended! Note: qgit is tricky to compile by hand, get a .deb or an .rpm
** [http://www.procode.org/stgit/ stgit] (StackedGIT is mainly for users doing heavy cherry picking. Only recommended for very advanced SCM users.)

Git is still developing quickly, but good mature versions are already available in packaged format. This guide is written with GIT v1.5.3 in mind.

== Downloading Moodle CVS History for git ==

There are currently two methods of retrieving Moodle CVS history for git:
* Clone the [http://www.catalyst.net.nz Catalyst IT] git import found at [http://git.catalyst.net.nz/gw?p=moodle-r2.git;a=summary git://git.catalyst.net.nz/moodle-r2.git]
* Run a local cvs import into git. ( Instructions for this can be found in [[Development:Importing Moodle CVS history into git]] )

It is '''highly advisable''' to clone the Catalyst git repository as this is updated hourly and the imported CVS history will share common commit hashes allowing developers to merge each others changes easily.

==Creating a Working Copy==

In order to use your repository you must clone yourself a working copy using [http://www.kernel.org/pub/software/scm/git/docs/git-clone.html git clone].

(Let's assume your destination directory is ~/src/moodle)

git clone git://git.catalyst.net.nz/moodle-r2.git ~/src/moodle

This will clone the catalyst moodle git repostitory into a local working copy that you may make changes to and commit to. You can then get updates from the catalyst git repository that can be merged in amongst the changes to your local copy.

==Using git effectively with Moodle==

Git can be used to intelligently track changes for creating client installs based off your own local, base, customisations which in turn tracks changes to the main Moodle CVS.

If you have a local git copy of the Moodle CVS repository you will have a number of heads such as: '''MOODLE_15_STABLE''', '''MOODLE_16_STABLE''', and so on. If you wish to create a new branch for your local customisations, you can base it off an existing HEAD (one of the CVS branches that exist in the main Moodle repository). If you wish to create a new local branch, '''mymoodle''', based off the current stable 1.6 code you would:

cd ~/src/moodle
git branch mymoodle MOODLE_16_STABLE
git branch ''# lists all the current branches, should show '''mymoodle'''
git checkout mymoodle ''# you are now working on the '''mymoodle''' branch''
''# add some new files (i.e. themes, blocks, etc.)''
git add .
git status ''# Will list your uncommited files''
git commit -m "Added base customisations."

To return to the '''master''' HEAD, equivalent to the main Moodle CVS HEAD branch, you would have to use:

git branch master

If you were running a nightly script to pull new Moodle CVS updates into your main git repository via git-cvsimport, you should also merge any new changes into your local branch:

cd ~/src/moodle.git
git checkout mymoodle
git merge MOODLE_16_STABLE

If you wanted to use this custom local branch to create a new client install, you could use:

git-clone ~src/moodle /home/someclient/public_html/
cd /home/someclient/public_html/
git checkout mymoodle origin/mymoodle

This would create a new git repository from your branch that could be used to do a client install which can, in turn, have the client's custom files added to it. This entire process can be automated so that your '''mymoodle''' branch is tracking upstream changes from the Moodle CVS ('''MOODLE_16_STABLE''' in this example) and your client installs are tracking changes against your local branch.

==Merging a patch from GIT into CVS==

If you have cvs access, you can use the [http://www.kernel.org/pub/software/scm/git/docs/git-cvsexportcommit.html git-cvsexportcommit] command to merge or cherry pick individual patches into ''upstream''.

::git-cvsexportcommit was initially written to do merges of NZOSVLE bugfixes into Moodle. Several tools and patches to GIT and Cogito come from the NZOSVLE team. [[User:Martin Langhoff|Martin Langhoff]] 14:55, 19 July 2006 (WST)

If the patches merge cleanly, future updates will recognise the already-applied patch. In that sense the merger in git is smarter than the merger in Cogito, consider using git-pull rather than cg-update. Note that if you see conflicts using git-pull you will have to work a bit harder to resolve them. Make sure you are familiar with handling merge conflicts with pure git before trying this.

==Preparing to merge large patch series or porting over to the next Moodle release==

When you have a large set of patches to apply to CVS, or a large set of custom patches to apply on top of the next release of Moodle, you can use [http://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html git-format-patch] to see what are your ''unmerged'' patches. git-format-patch tries hard to spot already-merged patches and skip them.

If you are applying patches to CVS, you can then use git-cvsexportcommit or plain old patch -p1 filename. If you are applying them to a new git branch (as you would to merge them on top of a new Moodle release) then you want to be on that new branch, and use git-am -3 -k filename.

If you are using git-am, make sure you learn how to deal with merge conflicts, and the use of git-update-index.

==Helpful Documentation==

* [http://www.kernel.org/pub/software/scm/git/docs/gitcvs-migration.html git for CVS users]
* [http://www.kernel.org/pub/software/scm/git/docs/git-cvsimport.html git-cvsimport man page]
* [http://www.kernel.org/pub/software/scm/git/docs/ git man pages]
* [http://vger.kernel.org/vger-lists.html The Git mailing list]
* [http://git.or.cz/gitwiki GitWiki]
* [http://www.kernel.org/git/ kernel.org Gitweb interface] (Browse the Git and associated tools' repositories)
* [http://wiki.sourcemage.org/Git_Guide Straightforward Git guide] (for Git >= 1.5)

==Moodle Forum Discussions==

* [http://moodle.org/mod/forum/discuss.php?d=91998 Tracking Moodle CVS with git (take2)]
* [http://moodle.org/mod/forum/discuss.php?d=42275 Tracking Moodle CVS with git]
* [http://moodle.org/mod/forum/discuss.php?d=34472 Merging Custom Moodle Code With Stable Releases]
* [http://moodle.org/mod/forum/discuss.php?d=66412 Moving to git, need serious help]
* [http://moodle.org/mod/forum/discuss.php?d=99763 The continuing story of moving Moodle towards Git]

[[Category:Developer|Tracking Moodle CVS with git]]

Development:Tracking Moodle CVS with git

2008-10-02T10:42:57Z

Poltawski: updated example directory

Note: Work in progress. These are notes for users that are comfortable with CVS and other SCMs, doing branching, merging, etc.

==Introduction==

If you plan on doing complicated or sustained development on Moodle, you will benefit from a local revision control system. Centralized systems like [http://www.nongnu.org/cvs/ CVS] and [http://subversion.tigris.org/ SVN] have limited capabilities for tracking vendor branches. For the ultimate experience you will want to use a system with distributed capabilities like [http://git.or.cz/ git]. Alternative tools for this include [http://www.selenic.com/mercurial/ Mercurial], [http://www.darcs.net/ Darcs] and [http://svk.elixus.org/ SVK].

GIT was developed by Linus Torvalds specifically for the Linux Kernel team. It is fast, fast, fast. Its usage is slightly different from CVS/SVN, but, if kernel developers can handle it ''Moodle developers will find it '''easy''' '' ;-)

Git is packaged for most major operating systems and an up to date list of packages can be found on [http://git.or.cz/ git.or.gz].

==Obtaining git==

You'll want:
* [http://www.kernel.org/pub/software/scm/git/ Git] (and its deps). It includes '''gitk''', a very good UI for visualising project history.
* an additional git porcelain (frontend)
** [http://sourceforge.net/projects/qgit qgit] a nice GUI to view the project history, make commits, etc. Recommended! Note: qgit is tricky to compile by hand, get a .deb or an .rpm
** [http://www.procode.org/stgit/ stgit] (StackedGIT is mainly for users doing heavy cherry picking. Only recommended for very advanced SCM users.)

Git is still developing quickly, but good mature versions are already available in packaged format. This guide is written with GIT v1.5.3 in mind.

== Downloading Moodle CVS History for git ==

There are currently two methods of retrieving Moodle CVS history for git:
* Clone the [http://www.catalyst.net.nz Catalyst IT] git import found at [http://git.catalyst.net.nz/gw?p=moodle-r2.git;a=summary git://git.catalyst.net.nz/moodle-r2.git]
* Run a local cvs import into git. ( Instructions for this can be found in [[Development:Importing Moodle CVS history into git]] )

It is '''highly advisable''' to clone the Catalyst git repository as this is updated hourly and the imported CVS history will share common commit hashes allowing developers to merge each others changes easily.

==Creating a Working Copy==

In order to use your repository you must clone yourself a working copy using [http://www.kernel.org/pub/software/scm/git/docs/git-clone.html git clone].

(Let's assume your destination directory above was ~/src/moodle)

git clone git://git.catalyst.net.nz/moodle-r2.git ~/src/moodle

This will clone the catalyst moodle git repostitory into a local working copy that you may make changes to and commit to. You can then get updates from the catalyst git repository that can be merged in amongst the changes to your local copy.

==Using git effectively with Moodle==

Git can be used to intelligently track changes for creating client installs based off your own local, base, customisations which in turn tracks changes to the main Moodle CVS.

If you have a local git copy of the Moodle CVS repository you will have a number of heads such as: '''MOODLE_15_STABLE''', '''MOODLE_16_STABLE''', and so on. If you wish to create a new branch for your local customisations, you can base it off an existing HEAD (one of the CVS branches that exist in the main Moodle repository). If you wish to create a new local branch, '''mymoodle''', based off the current stable 1.6 code you would:

cd ~/src/moodle
git branch mymoodle MOODLE_16_STABLE
git branch ''# lists all the current branches, should show '''mymoodle'''
git checkout mymoodle ''# you are now working on the '''mymoodle''' branch''
''# add some new files (i.e. themes, blocks, etc.)''
git add .
git status ''# Will list your uncommited files''
git commit -m "Added base customisations."

To return to the '''master''' HEAD, equivalent to the main Moodle CVS HEAD branch, you would have to use:

git branch master

If you were running a nightly script to pull new Moodle CVS updates into your main git repository via git-cvsimport, you should also merge any new changes into your local branch:

cd ~/src/moodle.git
git checkout mymoodle
git merge MOODLE_16_STABLE

If you wanted to use this custom local branch to create a new client install, you could use:

git-clone ~src/moodle /home/someclient/public_html/
cd /home/someclient/public_html/
git checkout mymoodle origin/mymoodle

This would create a new git repository from your branch that could be used to do a client install which can, in turn, have the client's custom files added to it. This entire process can be automated so that your '''mymoodle''' branch is tracking upstream changes from the Moodle CVS ('''MOODLE_16_STABLE''' in this example) and your client installs are tracking changes against your local branch.

==Merging a patch from GIT into CVS==

If you have cvs access, you can use the [http://www.kernel.org/pub/software/scm/git/docs/git-cvsexportcommit.html git-cvsexportcommit] command to merge or cherry pick individual patches into ''upstream''.

::git-cvsexportcommit was initially written to do merges of NZOSVLE bugfixes into Moodle. Several tools and patches to GIT and Cogito come from the NZOSVLE team. [[User:Martin Langhoff|Martin Langhoff]] 14:55, 19 July 2006 (WST)

If the patches merge cleanly, future updates will recognise the already-applied patch. In that sense the merger in git is smarter than the merger in Cogito, consider using git-pull rather than cg-update. Note that if you see conflicts using git-pull you will have to work a bit harder to resolve them. Make sure you are familiar with handling merge conflicts with pure git before trying this.

==Preparing to merge large patch series or porting over to the next Moodle release==

When you have a large set of patches to apply to CVS, or a large set of custom patches to apply on top of the next release of Moodle, you can use [http://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html git-format-patch] to see what are your ''unmerged'' patches. git-format-patch tries hard to spot already-merged patches and skip them.

If you are applying patches to CVS, you can then use git-cvsexportcommit or plain old patch -p1 filename. If you are applying them to a new git branch (as you would to merge them on top of a new Moodle release) then you want to be on that new branch, and use git-am -3 -k filename.

If you are using git-am, make sure you learn how to deal with merge conflicts, and the use of git-update-index.

==Helpful Documentation==

* [http://www.kernel.org/pub/software/scm/git/docs/gitcvs-migration.html git for CVS users]
* [http://www.kernel.org/pub/software/scm/git/docs/git-cvsimport.html git-cvsimport man page]
* [http://www.kernel.org/pub/software/scm/git/docs/ git man pages]
* [http://vger.kernel.org/vger-lists.html The Git mailing list]
* [http://git.or.cz/gitwiki GitWiki]
* [http://www.kernel.org/git/ kernel.org Gitweb interface] (Browse the Git and associated tools' repositories)
* [http://wiki.sourcemage.org/Git_Guide Straightforward Git guide] (for Git >= 1.5)

==Moodle Forum Discussions==

* [http://moodle.org/mod/forum/discuss.php?d=91998 Tracking Moodle CVS with git (take2)]
* [http://moodle.org/mod/forum/discuss.php?d=42275 Tracking Moodle CVS with git]
* [http://moodle.org/mod/forum/discuss.php?d=34472 Merging Custom Moodle Code With Stable Releases]
* [http://moodle.org/mod/forum/discuss.php?d=66412 Moving to git, need serious help]
* [http://moodle.org/mod/forum/discuss.php?d=99763 The continuing story of moving Moodle towards Git]

[[Category:Developer|Tracking Moodle CVS with git]]

Development:Tracking Moodle CVS with git

2008-10-02T10:41:00Z

Poltawski: updating example to use catalyst repo

Note: Work in progress. These are notes for users that are comfortable with CVS and other SCMs, doing branching, merging, etc.

==Introduction==

If you plan on doing complicated or sustained development on Moodle, you will benefit from a local revision control system. Centralized systems like [http://www.nongnu.org/cvs/ CVS] and [http://subversion.tigris.org/ SVN] have limited capabilities for tracking vendor branches. For the ultimate experience you will want to use a system with distributed capabilities like [http://git.or.cz/ git]. Alternative tools for this include [http://www.selenic.com/mercurial/ Mercurial], [http://www.darcs.net/ Darcs] and [http://svk.elixus.org/ SVK].

GIT was developed by Linus Torvalds specifically for the Linux Kernel team. It is fast, fast, fast. Its usage is slightly different from CVS/SVN, but, if kernel developers can handle it ''Moodle developers will find it '''easy''' '' ;-)

Git is packaged for most major operating systems and an up to date list of packages can be found on [http://git.or.cz/ git.or.gz].

==Obtaining git==

You'll want:
* [http://www.kernel.org/pub/software/scm/git/ Git] (and its deps). It includes '''gitk''', a very good UI for visualising project history.
* an additional git porcelain (frontend)
** [http://sourceforge.net/projects/qgit qgit] a nice GUI to view the project history, make commits, etc. Recommended! Note: qgit is tricky to compile by hand, get a .deb or an .rpm
** [http://www.procode.org/stgit/ stgit] (StackedGIT is mainly for users doing heavy cherry picking. Only recommended for very advanced SCM users.)

Git is still developing quickly, but good mature versions are already available in packaged format. This guide is written with GIT v1.5.3 in mind.

== Downloading Moodle CVS History for git ==

There are currently two methods of retrieving Moodle CVS history for git:
* Clone the [http://www.catalyst.net.nz Catalyst IT] git import found at [http://git.catalyst.net.nz/gw?p=moodle-r2.git;a=summary git://git.catalyst.net.nz/moodle-r2.git]
* Run a local cvs import into git. ( Instructions for this can be found in [[Development:Importing Moodle CVS history into git]] )

It is '''highly advisable''' to clone the Catalyst git repository as this is updated hourly and the imported CVS history will share common commit hashes allowing developers to merge each others changes easily.

==Creating a Working Copy==

In order to use your repository you must clone yourself a working copy using [http://www.kernel.org/pub/software/scm/git/docs/git-clone.html git clone].

(Let's assume your destination directory above was ~/src/moodle)

git clone git://git.catalyst.net.nz/moodle-r2.git ~/src/moodle

This will clone the catalyst moodle git repostitory into a local working copy that you may make changes to and commit to. You can then get updates from the catalyst git repository that can be merged in amongst the changes to your local copy.

==Using git effectively with Moodle==

Git can be used to intelligently track changes for creating client installs based off your own local, base, customisations which in turn tracks changes to the main Moodle CVS.

If you have a local git copy of the Moodle CVS repository you will have a number of heads such as: '''MOODLE_15_STABLE''', '''MOODLE_16_STABLE''', and so on. If you wish to create a new branch for your local customisations, you can base it off an existing HEAD (one of the CVS branches that exist in the main Moodle repository). If you wish to create a new local branch, '''mymoodle''', based off the current stable 1.6 code you would:

cd /pub/scm/moodle.git
git branch mymoodle MOODLE_16_STABLE
git branch ''# lists all the current branches, should show '''mymoodle'''
git checkout mymoodle ''# you are now working on the '''mymoodle''' branch''
''# add some new files (i.e. themes, blocks, etc.)''
git add .
git status ''# Will list your uncommited files''
git commit -m "Added base customisations."

To return to the '''master''' HEAD, equivalent to the main Moodle CVS HEAD branch, you would have to use:

git branch master

If you were running a nightly script to pull new Moodle CVS updates into your main git repository via git-cvsimport, you should also merge any new changes into your local branch:

cd /pub/scm/moodle.git
git checkout mymoodle
git merge MOODLE_16_STABLE

If you wanted to use this custom local branch to create a new client install, you could use:

git-clone /pub/scm/moodle.git#mymoodle /home/someclient/public_html/
cd /home/someclient/public_html/
git checkout mymoodle origin/mymoodle

This would create a new git repository from your branch that could be used to do a client install which can, in turn, have the client's custom files added to it. This entire process can be automated so that your '''mymoodle''' branch is tracking upstream changes from the Moodle CVS ('''MOODLE_16_STABLE''' in this example) and your client installs are tracking changes against your local branch.

==Merging a patch from GIT into CVS==

If you have cvs access, you can use the [http://www.kernel.org/pub/software/scm/git/docs/git-cvsexportcommit.html git-cvsexportcommit] command to merge or cherry pick individual patches into ''upstream''.

::git-cvsexportcommit was initially written to do merges of NZOSVLE bugfixes into Moodle. Several tools and patches to GIT and Cogito come from the NZOSVLE team. [[User:Martin Langhoff|Martin Langhoff]] 14:55, 19 July 2006 (WST)

If the patches merge cleanly, future updates will recognise the already-applied patch. In that sense the merger in git is smarter than the merger in Cogito, consider using git-pull rather than cg-update. Note that if you see conflicts using git-pull you will have to work a bit harder to resolve them. Make sure you are familiar with handling merge conflicts with pure git before trying this.

==Preparing to merge large patch series or porting over to the next Moodle release==

When you have a large set of patches to apply to CVS, or a large set of custom patches to apply on top of the next release of Moodle, you can use [http://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html git-format-patch] to see what are your ''unmerged'' patches. git-format-patch tries hard to spot already-merged patches and skip them.

If you are applying patches to CVS, you can then use git-cvsexportcommit or plain old patch -p1 filename. If you are applying them to a new git branch (as you would to merge them on top of a new Moodle release) then you want to be on that new branch, and use git-am -3 -k filename.

If you are using git-am, make sure you learn how to deal with merge conflicts, and the use of git-update-index.

==Helpful Documentation==

* [http://www.kernel.org/pub/software/scm/git/docs/gitcvs-migration.html git for CVS users]
* [http://www.kernel.org/pub/software/scm/git/docs/git-cvsimport.html git-cvsimport man page]
* [http://www.kernel.org/pub/software/scm/git/docs/ git man pages]
* [http://vger.kernel.org/vger-lists.html The Git mailing list]
* [http://git.or.cz/gitwiki GitWiki]
* [http://www.kernel.org/git/ kernel.org Gitweb interface] (Browse the Git and associated tools' repositories)
* [http://wiki.sourcemage.org/Git_Guide Straightforward Git guide] (for Git >= 1.5)

==Moodle Forum Discussions==

* [http://moodle.org/mod/forum/discuss.php?d=91998 Tracking Moodle CVS with git (take2)]
* [http://moodle.org/mod/forum/discuss.php?d=42275 Tracking Moodle CVS with git]
* [http://moodle.org/mod/forum/discuss.php?d=34472 Merging Custom Moodle Code With Stable Releases]
* [http://moodle.org/mod/forum/discuss.php?d=66412 Moving to git, need serious help]
* [http://moodle.org/mod/forum/discuss.php?d=99763 The continuing story of moving Moodle towards Git]

[[Category:Developer|Tracking Moodle CVS with git]]

Development:Tracking Moodle CVS with git

2008-10-02T10:33:39Z

Poltawski: note about catalyst repo (going to post on the forums to confirm this is ok later!)

Note: Work in progress. These are notes for users that are comfortable with CVS and other SCMs, doing branching, merging, etc.

==Introduction==

If you plan on doing complicated or sustained development on Moodle, you will benefit from a local revision control system. Centralized systems like [http://www.nongnu.org/cvs/ CVS] and [http://subversion.tigris.org/ SVN] have limited capabilities for tracking vendor branches. For the ultimate experience you will want to use a system with distributed capabilities like [http://git.or.cz/ git]. Alternative tools for this include [http://www.selenic.com/mercurial/ Mercurial], [http://www.darcs.net/ Darcs] and [http://svk.elixus.org/ SVK].

GIT was developed by Linus Torvalds specifically for the Linux Kernel team. It is fast, fast, fast. Its usage is slightly different from CVS/SVN, but, if kernel developers can handle it ''Moodle developers will find it '''easy''' '' ;-)

Git is packaged for most major operating systems and an up to date list of packages can be found on [http://git.or.cz/ git.or.gz].

==Obtaining git==

You'll want:
* [http://www.kernel.org/pub/software/scm/git/ Git] (and its deps). It includes '''gitk''', a very good UI for visualising project history.
* an additional git porcelain (frontend)
** [http://sourceforge.net/projects/qgit qgit] a nice GUI to view the project history, make commits, etc. Recommended! Note: qgit is tricky to compile by hand, get a .deb or an .rpm
** [http://www.procode.org/stgit/ stgit] (StackedGIT is mainly for users doing heavy cherry picking. Only recommended for very advanced SCM users.)

Git is still developing quickly, but good mature versions are already available in packaged format. This guide is written with GIT v1.5.3 in mind.

== Downloading Moodle CVS History for git ==

There are currently two methods of retrieving Moodle CVS history for git:
* Clone the [http://www.catalyst.net.nz Catalyst IT] git import found at [http://git.catalyst.net.nz/gw?p=moodle-r2.git;a=summary git://git.catalyst.net.nz/moodle-r2.git]
* Run a local cvs import into git. ( Instructions for this can be found in [[Development:Importing Moodle CVS history into git]] )

It is '''highly advisable''' to clone the Catalyst git repository as this is updated hourly and the imported CVS history will share common commit hashes allowing developers to merge each others changes easily.

==Creating a Working Copy==

In order to use your repository you must clone yourself a working copy using [http://www.kernel.org/pub/software/scm/git/docs/git-clone.html git-clone].

(Let's assume your destination directory above was /pub/scm/moodle)

mkdir ~/git/moodle
cd ~/git/moodle
git clone /pub/scm/moodle.git

This will clone the git filesystem you created from the CVS import into a working copy that you may make changes to and commit to. You can then get updates from the main git repository that will be merged in amongst the changes to your local copy.

==Using git effectively with Moodle==

Git can be used to intelligently track changes for creating client installs based off your own local, base, customisations which in turn tracks changes to the main Moodle CVS.

If you have a local git copy of the Moodle CVS repository you will have a number of heads such as: '''MOODLE_15_STABLE''', '''MOODLE_16_STABLE''', and so on. If you wish to create a new branch for your local customisations, you can base it off an existing HEAD (one of the CVS branches that exist in the main Moodle repository). If you wish to create a new local branch, '''mymoodle''', based off the current stable 1.6 code you would:

cd /pub/scm/moodle.git
git branch mymoodle MOODLE_16_STABLE
git branch ''# lists all the current branches, should show '''mymoodle'''
git checkout mymoodle ''# you are now working on the '''mymoodle''' branch''
''# add some new files (i.e. themes, blocks, etc.)''
git add .
git status ''# Will list your uncommited files''
git commit -m "Added base customisations."

To return to the '''master''' HEAD, equivalent to the main Moodle CVS HEAD branch, you would have to use:

git branch master

If you were running a nightly script to pull new Moodle CVS updates into your main git repository via git-cvsimport, you should also merge any new changes into your local branch:

cd /pub/scm/moodle.git
git checkout mymoodle
git merge MOODLE_16_STABLE

If you wanted to use this custom local branch to create a new client install, you could use:

git-clone /pub/scm/moodle.git#mymoodle /home/someclient/public_html/
cd /home/someclient/public_html/
git checkout mymoodle origin/mymoodle

This would create a new git repository from your branch that could be used to do a client install which can, in turn, have the client's custom files added to it. This entire process can be automated so that your '''mymoodle''' branch is tracking upstream changes from the Moodle CVS ('''MOODLE_16_STABLE''' in this example) and your client installs are tracking changes against your local branch.

==Merging a patch from GIT into CVS==

If you have cvs access, you can use the [http://www.kernel.org/pub/software/scm/git/docs/git-cvsexportcommit.html git-cvsexportcommit] command to merge or cherry pick individual patches into ''upstream''.

::git-cvsexportcommit was initially written to do merges of NZOSVLE bugfixes into Moodle. Several tools and patches to GIT and Cogito come from the NZOSVLE team. [[User:Martin Langhoff|Martin Langhoff]] 14:55, 19 July 2006 (WST)

If the patches merge cleanly, future updates will recognise the already-applied patch. In that sense the merger in git is smarter than the merger in Cogito, consider using git-pull rather than cg-update. Note that if you see conflicts using git-pull you will have to work a bit harder to resolve them. Make sure you are familiar with handling merge conflicts with pure git before trying this.

==Preparing to merge large patch series or porting over to the next Moodle release==

When you have a large set of patches to apply to CVS, or a large set of custom patches to apply on top of the next release of Moodle, you can use [http://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html git-format-patch] to see what are your ''unmerged'' patches. git-format-patch tries hard to spot already-merged patches and skip them.

If you are applying patches to CVS, you can then use git-cvsexportcommit or plain old patch -p1 filename. If you are applying them to a new git branch (as you would to merge them on top of a new Moodle release) then you want to be on that new branch, and use git-am -3 -k filename.

If you are using git-am, make sure you learn how to deal with merge conflicts, and the use of git-update-index.

==Helpful Documentation==

* [http://www.kernel.org/pub/software/scm/git/docs/gitcvs-migration.html git for CVS users]
* [http://www.kernel.org/pub/software/scm/git/docs/git-cvsimport.html git-cvsimport man page]
* [http://www.kernel.org/pub/software/scm/git/docs/ git man pages]
* [http://vger.kernel.org/vger-lists.html The Git mailing list]
* [http://git.or.cz/gitwiki GitWiki]
* [http://www.kernel.org/git/ kernel.org Gitweb interface] (Browse the Git and associated tools' repositories)
* [http://wiki.sourcemage.org/Git_Guide Straightforward Git guide] (for Git >= 1.5)

==Moodle Forum Discussions==

* [http://moodle.org/mod/forum/discuss.php?d=91998 Tracking Moodle CVS with git (take2)]
* [http://moodle.org/mod/forum/discuss.php?d=42275 Tracking Moodle CVS with git]
* [http://moodle.org/mod/forum/discuss.php?d=34472 Merging Custom Moodle Code With Stable Releases]
* [http://moodle.org/mod/forum/discuss.php?d=66412 Moving to git, need serious help]
* [http://moodle.org/mod/forum/discuss.php?d=99763 The continuing story of moving Moodle towards Git]

[[Category:Developer|Tracking Moodle CVS with git]]

Development:Tracking Moodle CVS with git

2008-10-02T10:19:51Z

Poltawski: added gitweb link

Note: Work in progress. These are notes for users that are comfortable with CVS and other SCMs, doing branching, merging, etc.

==Introduction==

If you plan on doing complicated or sustained development on Moodle, you will benefit from a local revision control system. Centralized systems like [http://www.nongnu.org/cvs/ CVS] and [http://subversion.tigris.org/ SVN] have limited capabilities for tracking vendor branches. For the ultimate experience you will want to use a system with distributed capabilities like [http://git.or.cz/ git]. Alternative tools for this include [http://www.selenic.com/mercurial/ Mercurial], [http://www.darcs.net/ Darcs] and [http://svk.elixus.org/ SVK].

GIT was developed by Linus Torvalds specifically for the Linux Kernel team. It is fast, fast, fast. Its usage is slightly different from CVS/SVN, but, if kernel developers can handle it ''Moodle developers will find it '''easy''' '' ;-)

Git is packaged for most major operating systems and an up to date list of packages can be found on [http://git.or.cz/ git.or.gz].

==Obtaining git==

You'll want:
* [http://www.kernel.org/pub/software/scm/git/ Git] (and its deps). It includes '''gitk''', a very good UI for visualising project history.
* an additional git porcelain (frontend)
** [http://sourceforge.net/projects/qgit qgit] a nice GUI to view the project history, make commits, etc. Recommended! Note: qgit is tricky to compile by hand, get a .deb or an .rpm
** [http://www.procode.org/stgit/ stgit] (StackedGIT is mainly for users doing heavy cherry picking. Only recommended for very advanced SCM users.)

Git is still developing quickly, but good mature versions are already available in packaged format. This guide is written with GIT v1.5.3 in mind.

== Downloading Moodle CVS History for git ==

There are currently two methods of retrieving Moodle CVS history for git:
* Clone the [http://www.catalyst.net.nz Catalyst IT] git import found at [http://git.catalyst.net.nz/gw?p=moodle-r2.git;a=summary git://git.catalyst.net.nz/moodle-r2.git]
* Run a local cvs import into git. ( Instructions for this can be found in [[Development:Importing Moodle CVS history into git]] )

==Creating a Working Copy==

In order to use your repository you must clone yourself a working copy using [http://www.kernel.org/pub/software/scm/git/docs/git-clone.html git-clone].

(Let's assume your destination directory above was /pub/scm/moodle)

mkdir ~/git/moodle
cd ~/git/moodle
git clone /pub/scm/moodle.git

This will clone the git filesystem you created from the CVS import into a working copy that you may make changes to and commit to. You can then get updates from the main git repository that will be merged in amongst the changes to your local copy.

==Using git effectively with Moodle==

Git can be used to intelligently track changes for creating client installs based off your own local, base, customisations which in turn tracks changes to the main Moodle CVS.

If you have a local git copy of the Moodle CVS repository you will have a number of heads such as: '''MOODLE_15_STABLE''', '''MOODLE_16_STABLE''', and so on. If you wish to create a new branch for your local customisations, you can base it off an existing HEAD (one of the CVS branches that exist in the main Moodle repository). If you wish to create a new local branch, '''mymoodle''', based off the current stable 1.6 code you would:

cd /pub/scm/moodle.git
git branch mymoodle MOODLE_16_STABLE
git branch ''# lists all the current branches, should show '''mymoodle'''
git checkout mymoodle ''# you are now working on the '''mymoodle''' branch''
''# add some new files (i.e. themes, blocks, etc.)''
git add .
git status ''# Will list your uncommited files''
git commit -m "Added base customisations."

To return to the '''master''' HEAD, equivalent to the main Moodle CVS HEAD branch, you would have to use:

git branch master

If you were running a nightly script to pull new Moodle CVS updates into your main git repository via git-cvsimport, you should also merge any new changes into your local branch:

cd /pub/scm/moodle.git
git checkout mymoodle
git merge MOODLE_16_STABLE

If you wanted to use this custom local branch to create a new client install, you could use:

git-clone /pub/scm/moodle.git#mymoodle /home/someclient/public_html/
cd /home/someclient/public_html/
git checkout mymoodle origin/mymoodle

This would create a new git repository from your branch that could be used to do a client install which can, in turn, have the client's custom files added to it. This entire process can be automated so that your '''mymoodle''' branch is tracking upstream changes from the Moodle CVS ('''MOODLE_16_STABLE''' in this example) and your client installs are tracking changes against your local branch.

==Merging a patch from GIT into CVS==

If you have cvs access, you can use the [http://www.kernel.org/pub/software/scm/git/docs/git-cvsexportcommit.html git-cvsexportcommit] command to merge or cherry pick individual patches into ''upstream''.

::git-cvsexportcommit was initially written to do merges of NZOSVLE bugfixes into Moodle. Several tools and patches to GIT and Cogito come from the NZOSVLE team. [[User:Martin Langhoff|Martin Langhoff]] 14:55, 19 July 2006 (WST)

If the patches merge cleanly, future updates will recognise the already-applied patch. In that sense the merger in git is smarter than the merger in Cogito, consider using git-pull rather than cg-update. Note that if you see conflicts using git-pull you will have to work a bit harder to resolve them. Make sure you are familiar with handling merge conflicts with pure git before trying this.

==Preparing to merge large patch series or porting over to the next Moodle release==

When you have a large set of patches to apply to CVS, or a large set of custom patches to apply on top of the next release of Moodle, you can use [http://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html git-format-patch] to see what are your ''unmerged'' patches. git-format-patch tries hard to spot already-merged patches and skip them.

If you are applying patches to CVS, you can then use git-cvsexportcommit or plain old patch -p1 filename. If you are applying them to a new git branch (as you would to merge them on top of a new Moodle release) then you want to be on that new branch, and use git-am -3 -k filename.

If you are using git-am, make sure you learn how to deal with merge conflicts, and the use of git-update-index.

==Helpful Documentation==

* [http://www.kernel.org/pub/software/scm/git/docs/gitcvs-migration.html git for CVS users]
* [http://www.kernel.org/pub/software/scm/git/docs/git-cvsimport.html git-cvsimport man page]
* [http://www.kernel.org/pub/software/scm/git/docs/ git man pages]
* [http://vger.kernel.org/vger-lists.html The Git mailing list]
* [http://git.or.cz/gitwiki GitWiki]
* [http://www.kernel.org/git/ kernel.org Gitweb interface] (Browse the Git and associated tools' repositories)
* [http://wiki.sourcemage.org/Git_Guide Straightforward Git guide] (for Git >= 1.5)

==Moodle Forum Discussions==

* [http://moodle.org/mod/forum/discuss.php?d=91998 Tracking Moodle CVS with git (take2)]
* [http://moodle.org/mod/forum/discuss.php?d=42275 Tracking Moodle CVS with git]
* [http://moodle.org/mod/forum/discuss.php?d=34472 Merging Custom Moodle Code With Stable Releases]
* [http://moodle.org/mod/forum/discuss.php?d=66412 Moving to git, need serious help]
* [http://moodle.org/mod/forum/discuss.php?d=99763 The continuing story of moving Moodle towards Git]

[[Category:Developer|Tracking Moodle CVS with git]]

Development:Tracking Moodle CVS with git

2008-10-02T10:19:19Z

Poltawski: removing git-cvsimport reference for this page

Note: Work in progress. These are notes for users that are comfortable with CVS and other SCMs, doing branching, merging, etc.

==Introduction==

If you plan on doing complicated or sustained development on Moodle, you will benefit from a local revision control system. Centralized systems like [http://www.nongnu.org/cvs/ CVS] and [http://subversion.tigris.org/ SVN] have limited capabilities for tracking vendor branches. For the ultimate experience you will want to use a system with distributed capabilities like [http://git.or.cz/ git]. Alternative tools for this include [http://www.selenic.com/mercurial/ Mercurial], [http://www.darcs.net/ Darcs] and [http://svk.elixus.org/ SVK].

GIT was developed by Linus Torvalds specifically for the Linux Kernel team. It is fast, fast, fast. Its usage is slightly different from CVS/SVN, but, if kernel developers can handle it ''Moodle developers will find it '''easy''' '' ;-)

Git is packaged for most major operating systems and an up to date list of packages can be found on [http://git.or.cz/ git.or.gz].

==Obtaining git==

You'll want:
* [http://www.kernel.org/pub/software/scm/git/ Git] (and its deps). It includes '''gitk''', a very good UI for visualising project history.
* an additional git porcelain (frontend)
** [http://sourceforge.net/projects/qgit qgit] a nice GUI to view the project history, make commits, etc. Recommended! Note: qgit is tricky to compile by hand, get a .deb or an .rpm
** [http://www.procode.org/stgit/ stgit] (StackedGIT is mainly for users doing heavy cherry picking. Only recommended for very advanced SCM users.)

Git is still developing quickly, but good mature versions are already available in packaged format. This guide is written with GIT v1.5.3 in mind.

== Downloading Moodle CVS History for git ==

There are currently two methods of retrieving Moodle CVS history for git:
* Clone the [http://www.catalyst.net.nz Catalyst IT] git import found at git://git.catalyst.net.nz/moodle-r2.git
* Run a local cvs import into git. ( Instructions for this can be found in [[Development:Importing Moodle CVS history into git]] )

==Creating a Working Copy==

In order to use your repository you must clone yourself a working copy using [http://www.kernel.org/pub/software/scm/git/docs/git-clone.html git-clone].

(Let's assume your destination directory above was /pub/scm/moodle)

mkdir ~/git/moodle
cd ~/git/moodle
git clone /pub/scm/moodle.git

This will clone the git filesystem you created from the CVS import into a working copy that you may make changes to and commit to. You can then get updates from the main git repository that will be merged in amongst the changes to your local copy.

==Using git effectively with Moodle==

Git can be used to intelligently track changes for creating client installs based off your own local, base, customisations which in turn tracks changes to the main Moodle CVS.

If you have a local git copy of the Moodle CVS repository you will have a number of heads such as: '''MOODLE_15_STABLE''', '''MOODLE_16_STABLE''', and so on. If you wish to create a new branch for your local customisations, you can base it off an existing HEAD (one of the CVS branches that exist in the main Moodle repository). If you wish to create a new local branch, '''mymoodle''', based off the current stable 1.6 code you would:

cd /pub/scm/moodle.git
git branch mymoodle MOODLE_16_STABLE
git branch ''# lists all the current branches, should show '''mymoodle'''
git checkout mymoodle ''# you are now working on the '''mymoodle''' branch''
''# add some new files (i.e. themes, blocks, etc.)''
git add .
git status ''# Will list your uncommited files''
git commit -m "Added base customisations."

To return to the '''master''' HEAD, equivalent to the main Moodle CVS HEAD branch, you would have to use:

git branch master

If you were running a nightly script to pull new Moodle CVS updates into your main git repository via git-cvsimport, you should also merge any new changes into your local branch:

cd /pub/scm/moodle.git
git checkout mymoodle
git merge MOODLE_16_STABLE

If you wanted to use this custom local branch to create a new client install, you could use:

git-clone /pub/scm/moodle.git#mymoodle /home/someclient/public_html/
cd /home/someclient/public_html/
git checkout mymoodle origin/mymoodle

This would create a new git repository from your branch that could be used to do a client install which can, in turn, have the client's custom files added to it. This entire process can be automated so that your '''mymoodle''' branch is tracking upstream changes from the Moodle CVS ('''MOODLE_16_STABLE''' in this example) and your client installs are tracking changes against your local branch.

==Merging a patch from GIT into CVS==

If you have cvs access, you can use the [http://www.kernel.org/pub/software/scm/git/docs/git-cvsexportcommit.html git-cvsexportcommit] command to merge or cherry pick individual patches into ''upstream''.

::git-cvsexportcommit was initially written to do merges of NZOSVLE bugfixes into Moodle. Several tools and patches to GIT and Cogito come from the NZOSVLE team. [[User:Martin Langhoff|Martin Langhoff]] 14:55, 19 July 2006 (WST)

If the patches merge cleanly, future updates will recognise the already-applied patch. In that sense the merger in git is smarter than the merger in Cogito, consider using git-pull rather than cg-update. Note that if you see conflicts using git-pull you will have to work a bit harder to resolve them. Make sure you are familiar with handling merge conflicts with pure git before trying this.

==Preparing to merge large patch series or porting over to the next Moodle release==

When you have a large set of patches to apply to CVS, or a large set of custom patches to apply on top of the next release of Moodle, you can use [http://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html git-format-patch] to see what are your ''unmerged'' patches. git-format-patch tries hard to spot already-merged patches and skip them.

If you are applying patches to CVS, you can then use git-cvsexportcommit or plain old patch -p1 filename. If you are applying them to a new git branch (as you would to merge them on top of a new Moodle release) then you want to be on that new branch, and use git-am -3 -k filename.

If you are using git-am, make sure you learn how to deal with merge conflicts, and the use of git-update-index.

==Helpful Documentation==

* [http://www.kernel.org/pub/software/scm/git/docs/gitcvs-migration.html git for CVS users]
* [http://www.kernel.org/pub/software/scm/git/docs/git-cvsimport.html git-cvsimport man page]
* [http://www.kernel.org/pub/software/scm/git/docs/ git man pages]
* [http://vger.kernel.org/vger-lists.html The Git mailing list]
* [http://git.or.cz/gitwiki GitWiki]
* [http://www.kernel.org/git/ kernel.org Gitweb interface] (Browse the Git and associated tools' repositories)
* [http://wiki.sourcemage.org/Git_Guide Straightforward Git guide] (for Git >= 1.5)

==Moodle Forum Discussions==

* [http://moodle.org/mod/forum/discuss.php?d=91998 Tracking Moodle CVS with git (take2)]
* [http://moodle.org/mod/forum/discuss.php?d=42275 Tracking Moodle CVS with git]
* [http://moodle.org/mod/forum/discuss.php?d=34472 Merging Custom Moodle Code With Stable Releases]
* [http://moodle.org/mod/forum/discuss.php?d=66412 Moving to git, need serious help]
* [http://moodle.org/mod/forum/discuss.php?d=99763 The continuing story of moving Moodle towards Git]

[[Category:Developer|Tracking Moodle CVS with git]]

Development:Importing Moodle CVS history into git

2008-10-02T10:16:56Z

Poltawski: added category

==Importing CVS==

To begin we must import the CVS repository into a local git filesystem using [http://www.kernel.org/pub/software/scm/git/docs/git-cvsimport.html git-cvsimport].

#!/bin/bash
CVSROOT=:pserver:anonymous@uk.cvs.moodle.org:/cvsroot/moodle
MODULE=moodle
INSTALLDIR=moodle
git-cvsimport -p x -v -k -o cvshead -d $CVSROOT -C $INSTALLDIR $MODULE &> cvsimport.log

(When might one want to use the -m -u and -s flags?)
-u purely cosmetic
-s slashes can cause problems but this doesn't seem to be the case with moodle
-m ?

This will run for a '''very''' long time as it checks out every revision of each file (including all branches). However, if you run the command after a successful initial run it will simply get the new revisions since it was last run. This creates a new git repository which can then be cloned and worked upon.

At the time of writing the size of a new Moodle git repository was approximately 866MB ''unpacked'', and 75MB packed. Newer versions of git pack the project during import, so your resulting repository plus checkout should be a bit over 100MB. Still, it is a good idea to pack the repository to make it smaller and faster, running

git-repack -a -d

With the -o cvshead flag, the "HEAD" of the CVS repo will appear as 'cvshead' in GIT, making things a lot clearer.

==Importing CVS Faster==

For initial imports, it is highly recommended that you fetch the CVS repository from one of Moodle [[CVS_for_Administrators#CVS_Servers | cvs servers]] and run the initial cvsimport against your local copy of the repository.

:: For the more adventurous: Keith Packard wrote another importer that does a better job at the initial import, called parsecvs. I use parsecvs for the initial import and then git-cvsimport daily for the incremental imports. --[[User:Martin Langhoff|Martin Langhoff]] 15:04, 19 July 2006 (WST)

:: It took me a few minutes to find the info in the Sourceforge docs on how to fetch the raw CVS repo via rsync. Just type 'rsync -av "rsync://moodle.cvs.sourceforge.net/cvsroot/moodle/moodle/*" .' (without the single quotes) and you'll be done --[[User:Iñaki Arenaza|Iñaki Arenaza]] 17:54, 25 December 2006 (CST).

:: This rsync command line doesn't work as-is anymore, since the repository was moved from sourceforge. Some mirrors (at least es.cvs.moodle.org at this time) provide rsync, but some don't (like eu.cvs.moodle.org). Find the best mirror for you and update the rsync call accordingly.

After the initial import, you can run git-cvsimport against your favorite cvs.moodle.org mirror repository, or update your local copy via rsync as sas demonstrated in [http://www.progsoc.org/~wildfire/git/update-repo.sh this script].

[[Category:Developer|Importing Moodle CVS history into git]]

Development:Tracking Moodle CVS with git

2008-10-02T10:16:04Z

Poltawski: /* Downloading Moodle CVS History for git */

Note: Work in progress. These are notes for users that are comfortable with CVS and other SCMs, doing branching, merging, etc.

==Introduction==

If you plan on doing complicated or sustained development on Moodle, you will benefit from a local revision control system. Centralized systems like [http://www.nongnu.org/cvs/ CVS] and [http://subversion.tigris.org/ SVN] have limited capabilities for tracking vendor branches. For the ultimate experience you will want to use a system with distributed capabilities like [http://git.or.cz/ git]. Alternative tools for this include [http://www.selenic.com/mercurial/ Mercurial], [http://www.darcs.net/ Darcs] and [http://svk.elixus.org/ SVK].

GIT was developed by Linus Torvalds specifically for the Linux Kernel team. It is fast, fast, fast. Its usage is slightly different from CVS/SVN, but, if kernel developers can handle it ''Moodle developers will find it '''easy''' '' ;-)

Git is packaged for most major operating systems and an up to date list of packages can be found on [http://git.or.cz/ git.or.gz].

==Obtaining git==

You'll want:
* [http://www.kernel.org/pub/software/scm/git/ Git] (and its deps). It includes '''gitk''', a very good UI for visualising project history.
* [http://www.cobite.com/cvsps/ cvsps] (a dependency of git-cvsimport make sure you have the latest release)
* an additional git porcelain (frontend)
** [http://sourceforge.net/projects/qgit qgit] a nice GUI to view the project history, make commits, etc. Recommended! Note: qgit is tricky to compile by hand, get a .deb or an .rpm
** [http://www.procode.org/stgit/ stgit] (StackedGIT is mainly for users doing heavy cherry picking. Only recommended for very advanced SCM users.)

Git is still developing quickly, but good mature versions are already available in packaged format. This guide is written with GIT v1.5.3 in mind.

== Downloading Moodle CVS History for git ==

There are currently two methods of retrieving Moodle CVS history for git:
* Clone the [http://www.catalyst.net.nz Catalyst IT] git import found at git://git.catalyst.net.nz/moodle-r2.git
* Run a local cvs import into git. ( Instructions for this can be found in [[Development:Importing Moodle CVS history into git]] )

==Creating a Working Copy==

In order to use your repository you must clone yourself a working copy using [http://www.kernel.org/pub/software/scm/git/docs/git-clone.html git-clone].

(Let's assume your destination directory above was /pub/scm/moodle)

mkdir ~/git/moodle
cd ~/git/moodle
git clone /pub/scm/moodle.git

This will clone the git filesystem you created from the CVS import into a working copy that you may make changes to and commit to. You can then get updates from the main git repository that will be merged in amongst the changes to your local copy.

==Using git effectively with Moodle==

Git can be used to intelligently track changes for creating client installs based off your own local, base, customisations which in turn tracks changes to the main Moodle CVS.

If you have a local git copy of the Moodle CVS repository you will have a number of heads such as: '''MOODLE_15_STABLE''', '''MOODLE_16_STABLE''', and so on. If you wish to create a new branch for your local customisations, you can base it off an existing HEAD (one of the CVS branches that exist in the main Moodle repository). If you wish to create a new local branch, '''mymoodle''', based off the current stable 1.6 code you would:

cd /pub/scm/moodle.git
git branch mymoodle MOODLE_16_STABLE
git branch ''# lists all the current branches, should show '''mymoodle'''
git checkout mymoodle ''# you are now working on the '''mymoodle''' branch''
''# add some new files (i.e. themes, blocks, etc.)''
git add .
git status ''# Will list your uncommited files''
git commit -m "Added base customisations."

To return to the '''master''' HEAD, equivalent to the main Moodle CVS HEAD branch, you would have to use:

git branch master

If you were running a nightly script to pull new Moodle CVS updates into your main git repository via git-cvsimport, you should also merge any new changes into your local branch:

cd /pub/scm/moodle.git
git checkout mymoodle
git merge MOODLE_16_STABLE

If you wanted to use this custom local branch to create a new client install, you could use:

git-clone /pub/scm/moodle.git#mymoodle /home/someclient/public_html/
cd /home/someclient/public_html/
git checkout mymoodle origin/mymoodle

This would create a new git repository from your branch that could be used to do a client install which can, in turn, have the client's custom files added to it. This entire process can be automated so that your '''mymoodle''' branch is tracking upstream changes from the Moodle CVS ('''MOODLE_16_STABLE''' in this example) and your client installs are tracking changes against your local branch.

==Merging a patch from GIT into CVS==

If you have cvs access, you can use the [http://www.kernel.org/pub/software/scm/git/docs/git-cvsexportcommit.html git-cvsexportcommit] command to merge or cherry pick individual patches into ''upstream''.

::git-cvsexportcommit was initially written to do merges of NZOSVLE bugfixes into Moodle. Several tools and patches to GIT and Cogito come from the NZOSVLE team. [[User:Martin Langhoff|Martin Langhoff]] 14:55, 19 July 2006 (WST)

If the patches merge cleanly, future updates will recognise the already-applied patch. In that sense the merger in git is smarter than the merger in Cogito, consider using git-pull rather than cg-update. Note that if you see conflicts using git-pull you will have to work a bit harder to resolve them. Make sure you are familiar with handling merge conflicts with pure git before trying this.

==Preparing to merge large patch series or porting over to the next Moodle release==

When you have a large set of patches to apply to CVS, or a large set of custom patches to apply on top of the next release of Moodle, you can use [http://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html git-format-patch] to see what are your ''unmerged'' patches. git-format-patch tries hard to spot already-merged patches and skip them.

If you are applying patches to CVS, you can then use git-cvsexportcommit or plain old patch -p1 filename. If you are applying them to a new git branch (as you would to merge them on top of a new Moodle release) then you want to be on that new branch, and use git-am -3 -k filename.

If you are using git-am, make sure you learn how to deal with merge conflicts, and the use of git-update-index.

==Helpful Documentation==

* [http://www.kernel.org/pub/software/scm/git/docs/gitcvs-migration.html git for CVS users]
* [http://www.kernel.org/pub/software/scm/git/docs/git-cvsimport.html git-cvsimport man page]
* [http://www.kernel.org/pub/software/scm/git/docs/ git man pages]
* [http://vger.kernel.org/vger-lists.html The Git mailing list]
* [http://git.or.cz/gitwiki GitWiki]
* [http://www.kernel.org/git/ kernel.org Gitweb interface] (Browse the Git and associated tools' repositories)
* [http://wiki.sourcemage.org/Git_Guide Straightforward Git guide] (for Git >= 1.5)

==Moodle Forum Discussions==

* [http://moodle.org/mod/forum/discuss.php?d=91998 Tracking Moodle CVS with git (take2)]
* [http://moodle.org/mod/forum/discuss.php?d=42275 Tracking Moodle CVS with git]
* [http://moodle.org/mod/forum/discuss.php?d=34472 Merging Custom Moodle Code With Stable Releases]
* [http://moodle.org/mod/forum/discuss.php?d=66412 Moving to git, need serious help]
* [http://moodle.org/mod/forum/discuss.php?d=99763 The continuing story of moving Moodle towards Git]

[[Category:Developer|Tracking Moodle CVS with git]]

Development:Tracking Moodle CVS with git

2008-10-02T10:15:43Z

Poltawski: I have split the cvs import section into [Development:Importing Moodle CVS history into git] . Accepted wisdom seems to be that we should just all use the catalyst repo these days ;)

Note: Work in progress. These are notes for users that are comfortable with CVS and other SCMs, doing branching, merging, etc.

==Introduction==

If you plan on doing complicated or sustained development on Moodle, you will benefit from a local revision control system. Centralized systems like [http://www.nongnu.org/cvs/ CVS] and [http://subversion.tigris.org/ SVN] have limited capabilities for tracking vendor branches. For the ultimate experience you will want to use a system with distributed capabilities like [http://git.or.cz/ git]. Alternative tools for this include [http://www.selenic.com/mercurial/ Mercurial], [http://www.darcs.net/ Darcs] and [http://svk.elixus.org/ SVK].

GIT was developed by Linus Torvalds specifically for the Linux Kernel team. It is fast, fast, fast. Its usage is slightly different from CVS/SVN, but, if kernel developers can handle it ''Moodle developers will find it '''easy''' '' ;-)

Git is packaged for most major operating systems and an up to date list of packages can be found on [http://git.or.cz/ git.or.gz].

==Obtaining git==

You'll want:
* [http://www.kernel.org/pub/software/scm/git/ Git] (and its deps). It includes '''gitk''', a very good UI for visualising project history.
* [http://www.cobite.com/cvsps/ cvsps] (a dependency of git-cvsimport make sure you have the latest release)
* an additional git porcelain (frontend)
** [http://sourceforge.net/projects/qgit qgit] a nice GUI to view the project history, make commits, etc. Recommended! Note: qgit is tricky to compile by hand, get a .deb or an .rpm
** [http://www.procode.org/stgit/ stgit] (StackedGIT is mainly for users doing heavy cherry picking. Only recommended for very advanced SCM users.)

Git is still developing quickly, but good mature versions are already available in packaged format. This guide is written with GIT v1.5.3 in mind.

== Downloading Moodle CVS History for git ==

There are currently two methods of retrieving Moodle CVS history for git:
* Clone the [http://www.catalyst.net.nz Catalyst IT] git import found at git://git.catalyst.net.nz/moodle-r2.git
* Run your a local cvs import into git. ( Instructions for this can be found in [[Development:Importing Moodle CVS history into git]] )

==Creating a Working Copy==

In order to use your repository you must clone yourself a working copy using [http://www.kernel.org/pub/software/scm/git/docs/git-clone.html git-clone].

(Let's assume your destination directory above was /pub/scm/moodle)

mkdir ~/git/moodle
cd ~/git/moodle
git clone /pub/scm/moodle.git

This will clone the git filesystem you created from the CVS import into a working copy that you may make changes to and commit to. You can then get updates from the main git repository that will be merged in amongst the changes to your local copy.

==Using git effectively with Moodle==

Git can be used to intelligently track changes for creating client installs based off your own local, base, customisations which in turn tracks changes to the main Moodle CVS.

If you have a local git copy of the Moodle CVS repository you will have a number of heads such as: '''MOODLE_15_STABLE''', '''MOODLE_16_STABLE''', and so on. If you wish to create a new branch for your local customisations, you can base it off an existing HEAD (one of the CVS branches that exist in the main Moodle repository). If you wish to create a new local branch, '''mymoodle''', based off the current stable 1.6 code you would:

cd /pub/scm/moodle.git
git branch mymoodle MOODLE_16_STABLE
git branch ''# lists all the current branches, should show '''mymoodle'''
git checkout mymoodle ''# you are now working on the '''mymoodle''' branch''
''# add some new files (i.e. themes, blocks, etc.)''
git add .
git status ''# Will list your uncommited files''
git commit -m "Added base customisations."

To return to the '''master''' HEAD, equivalent to the main Moodle CVS HEAD branch, you would have to use:

git branch master

If you were running a nightly script to pull new Moodle CVS updates into your main git repository via git-cvsimport, you should also merge any new changes into your local branch:

cd /pub/scm/moodle.git
git checkout mymoodle
git merge MOODLE_16_STABLE

If you wanted to use this custom local branch to create a new client install, you could use:

git-clone /pub/scm/moodle.git#mymoodle /home/someclient/public_html/
cd /home/someclient/public_html/
git checkout mymoodle origin/mymoodle

This would create a new git repository from your branch that could be used to do a client install which can, in turn, have the client's custom files added to it. This entire process can be automated so that your '''mymoodle''' branch is tracking upstream changes from the Moodle CVS ('''MOODLE_16_STABLE''' in this example) and your client installs are tracking changes against your local branch.

==Merging a patch from GIT into CVS==

If you have cvs access, you can use the [http://www.kernel.org/pub/software/scm/git/docs/git-cvsexportcommit.html git-cvsexportcommit] command to merge or cherry pick individual patches into ''upstream''.

::git-cvsexportcommit was initially written to do merges of NZOSVLE bugfixes into Moodle. Several tools and patches to GIT and Cogito come from the NZOSVLE team. [[User:Martin Langhoff|Martin Langhoff]] 14:55, 19 July 2006 (WST)

If the patches merge cleanly, future updates will recognise the already-applied patch. In that sense the merger in git is smarter than the merger in Cogito, consider using git-pull rather than cg-update. Note that if you see conflicts using git-pull you will have to work a bit harder to resolve them. Make sure you are familiar with handling merge conflicts with pure git before trying this.

==Preparing to merge large patch series or porting over to the next Moodle release==

When you have a large set of patches to apply to CVS, or a large set of custom patches to apply on top of the next release of Moodle, you can use [http://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html git-format-patch] to see what are your ''unmerged'' patches. git-format-patch tries hard to spot already-merged patches and skip them.

If you are applying patches to CVS, you can then use git-cvsexportcommit or plain old patch -p1 filename. If you are applying them to a new git branch (as you would to merge them on top of a new Moodle release) then you want to be on that new branch, and use git-am -3 -k filename.

If you are using git-am, make sure you learn how to deal with merge conflicts, and the use of git-update-index.

==Helpful Documentation==

* [http://www.kernel.org/pub/software/scm/git/docs/gitcvs-migration.html git for CVS users]
* [http://www.kernel.org/pub/software/scm/git/docs/git-cvsimport.html git-cvsimport man page]
* [http://www.kernel.org/pub/software/scm/git/docs/ git man pages]
* [http://vger.kernel.org/vger-lists.html The Git mailing list]
* [http://git.or.cz/gitwiki GitWiki]
* [http://www.kernel.org/git/ kernel.org Gitweb interface] (Browse the Git and associated tools' repositories)
* [http://wiki.sourcemage.org/Git_Guide Straightforward Git guide] (for Git >= 1.5)

==Moodle Forum Discussions==

* [http://moodle.org/mod/forum/discuss.php?d=91998 Tracking Moodle CVS with git (take2)]
* [http://moodle.org/mod/forum/discuss.php?d=42275 Tracking Moodle CVS with git]
* [http://moodle.org/mod/forum/discuss.php?d=34472 Merging Custom Moodle Code With Stable Releases]
* [http://moodle.org/mod/forum/discuss.php?d=66412 Moving to git, need serious help]
* [http://moodle.org/mod/forum/discuss.php?d=99763 The continuing story of moving Moodle towards Git]

[[Category:Developer|Tracking Moodle CVS with git]]

Development:Importing Moodle CVS history into git

2008-10-02T09:59:29Z

Poltawski: Moved this content from existing page: Development:Tracking Moodle CVS with git

Development:Tracking Moodle CVS with git

2008-10-01T22:38:12Z

Poltawski: remove note about cogito, git 1.5.3 offers good interface

Note: Work in progress. These are notes for users that are comfortable with CVS and other SCMs, doing branching, merging, etc.

==Introduction==

If you plan on doing complicated or sustained development on Moodle, you will benefit from a local revision control system. Centralized systems like [http://www.nongnu.org/cvs/ CVS] and [http://subversion.tigris.org/ SVN] have limited capabilities for tracking vendor branches. For the ultimate experience you will want to use a system with distributed capabilities like [http://git.or.cz/ git]. Alternative tools for this include [http://www.selenic.com/mercurial/ Mercurial], [http://www.darcs.net/ Darcs] and [http://svk.elixus.org/ SVK].

GIT was developed by Linus Torvalds specifically for the Linux Kernel team. It is fast, fast, fast. Its usage is slightly different from CVS/SVN, but, if kernel developers can handle it ''Moodle developers will find it '''easy''' '' ;-)

Git is packaged for most major operating systems and an up to date list of packages can be found on [http://git.or.cz/ git.or.gz].

==Obtaining git==

You'll want:
* [http://www.kernel.org/pub/software/scm/git/ Git] (and its deps). It includes '''gitk''', a very good UI for visualising project history.
* [http://www.cobite.com/cvsps/ cvsps] (a dependency of git-cvsimport make sure you have the latest release)
* an additional git porcelain (frontend)
** [http://sourceforge.net/projects/qgit qgit] a nice GUI to view the project history, make commits, etc. Recommended! Note: qgit is tricky to compile by hand, get a .deb or an .rpm
** [http://www.procode.org/stgit/ stgit] (StackedGIT is mainly for users doing heavy cherry picking. Only recommended for very advanced SCM users.)

Git is still developing quickly, but good mature versions are already available in packaged format. This guide is written with GIT v1.5.3 in mind.

==Importing CVS==

To begin we must import the CVS repository into a local git filesystem using [http://www.kernel.org/pub/software/scm/git/docs/git-cvsimport.html git-cvsimport].

#!/bin/bash
CVSROOT=:pserver:anonymous@uk.cvs.moodle.org:/cvsroot/moodle
MODULE=moodle
INSTALLDIR=moodle
git-cvsimport -p x -v -k -o cvshead -d $CVSROOT -C $INSTALLDIR $MODULE &> cvsimport.log

(When might one want to use the -m -u and -s flags?)
-u purely cosmetic
-s slashes can cause problems but this doesn't seem to be the case with moodle
-m ?

This will run for a '''very''' long time as it checks out every revision of each file (including all branches). However, if you run the command after a successful initial run it will simply get the new revisions since it was last run. This creates a new git repository which can then be cloned and worked upon.

At the time of writing the size of a new Moodle git repository was approximately 866MB ''unpacked'', and 75MB packed. Newer versions of git pack the project during import, so your resulting repository plus checkout should be a bit over 100MB. Still, it is a good idea to pack the repository to make it smaller and faster, running

git-repack -a -d

With the -o cvshead flag, the "HEAD" of the CVS repo will appear as 'cvshead' in GIT, making things a lot clearer.

==Importing CVS Faster==

For initial imports, it is highly recommended that you fetch the CVS repository from one of Moodle [[CVS_for_Administrators#CVS_Servers | cvs servers]] and run the initial cvsimport against your local copy of the repository.

:: For the more adventurous: Keith Packard wrote another importer that does a better job at the initial import, called parsecvs. I use parsecvs for the initial import and then git-cvsimport daily for the incremental imports. --[[User:Martin Langhoff|Martin Langhoff]] 15:04, 19 July 2006 (WST)

:: It took me a few minutes to find the info in the Sourceforge docs on how to fetch the raw CVS repo via rsync. Just type 'rsync -av "rsync://moodle.cvs.sourceforge.net/cvsroot/moodle/moodle/*" .' (without the single quotes) and you'll be done --[[User:Iñaki Arenaza|Iñaki Arenaza]] 17:54, 25 December 2006 (CST).

:: This rsync command line doesn't work as-is anymore, since the repository was moved from sourceforge. Some mirrors (at least es.cvs.moodle.org at this time) provide rsync, but some don't (like eu.cvs.moodle.org). Find the best mirror for you and update the rsync call accordingly.

After the initial import, you can run git-cvsimport against your favorite cvs.moodle.org mirror repository, or update your local copy via rsync as sas demonstrated in [http://www.progsoc.org/~wildfire/git/update-repo.sh this script].

==Creating a Working Copy==

In order to use your repository you must clone yourself a working copy using [http://www.kernel.org/pub/software/scm/git/docs/git-clone.html git-clone].

(Let's assume your destination directory above was /pub/scm/moodle)

mkdir ~/git/moodle
cd ~/git/moodle
git clone /pub/scm/moodle.git

This will clone the git filesystem you created from the CVS import into a working copy that you may make changes to and commit to. You can then get updates from the main git repository that will be merged in amongst the changes to your local copy.

==Using git effectively with Moodle==

Git can be used to intelligently track changes for creating client installs based off your own local, base, customisations which in turn tracks changes to the main Moodle CVS.

If you have a local git copy of the Moodle CVS repository you will have a number of heads such as: '''MOODLE_15_STABLE''', '''MOODLE_16_STABLE''', and so on. If you wish to create a new branch for your local customisations, you can base it off an existing HEAD (one of the CVS branches that exist in the main Moodle repository). If you wish to create a new local branch, '''mymoodle''', based off the current stable 1.6 code you would:

cd /pub/scm/moodle.git
git branch mymoodle MOODLE_16_STABLE
git branch ''# lists all the current branches, should show '''mymoodle'''
git checkout mymoodle ''# you are now working on the '''mymoodle''' branch''
''# add some new files (i.e. themes, blocks, etc.)''
git add .
git status ''# Will list your uncommited files''
git commit -m "Added base customisations."

To return to the '''master''' HEAD, equivalent to the main Moodle CVS HEAD branch, you would have to use:

git branch master

If you were running a nightly script to pull new Moodle CVS updates into your main git repository via git-cvsimport, you should also merge any new changes into your local branch:

cd /pub/scm/moodle.git
git checkout mymoodle
git merge MOODLE_16_STABLE

If you wanted to use this custom local branch to create a new client install, you could use:

git-clone /pub/scm/moodle.git#mymoodle /home/someclient/public_html/
cd /home/someclient/public_html/
git checkout mymoodle origin/mymoodle

This would create a new git repository from your branch that could be used to do a client install which can, in turn, have the client's custom files added to it. This entire process can be automated so that your '''mymoodle''' branch is tracking upstream changes from the Moodle CVS ('''MOODLE_16_STABLE''' in this example) and your client installs are tracking changes against your local branch.

==Merging a patch from GIT into CVS==

If you have cvs access, you can use the [http://www.kernel.org/pub/software/scm/git/docs/git-cvsexportcommit.html git-cvsexportcommit] command to merge or cherry pick individual patches into ''upstream''.

::git-cvsexportcommit was initially written to do merges of NZOSVLE bugfixes into Moodle. Several tools and patches to GIT and Cogito come from the NZOSVLE team. [[User:Martin Langhoff|Martin Langhoff]] 14:55, 19 July 2006 (WST)

If the patches merge cleanly, future updates will recognise the already-applied patch. In that sense the merger in git is smarter than the merger in Cogito, consider using git-pull rather than cg-update. Note that if you see conflicts using git-pull you will have to work a bit harder to resolve them. Make sure you are familiar with handling merge conflicts with pure git before trying this.

==Preparing to merge large patch series or porting over to the next Moodle release==

When you have a large set of patches to apply to CVS, or a large set of custom patches to apply on top of the next release of Moodle, you can use [http://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html git-format-patch] to see what are your ''unmerged'' patches. git-format-patch tries hard to spot already-merged patches and skip them.

If you are applying patches to CVS, you can then use git-cvsexportcommit or plain old patch -p1 filename. If you are applying them to a new git branch (as you would to merge them on top of a new Moodle release) then you want to be on that new branch, and use git-am -3 -k filename.

If you are using git-am, make sure you learn how to deal with merge conflicts, and the use of git-update-index.

==Helpful Documentation==

* [http://www.kernel.org/pub/software/scm/git/docs/gitcvs-migration.html git for CVS users]
* [http://www.kernel.org/pub/software/scm/git/docs/git-cvsimport.html git-cvsimport man page]
* [http://www.kernel.org/pub/software/scm/git/docs/ git man pages]
* [http://vger.kernel.org/vger-lists.html The Git mailing list]
* [http://git.or.cz/gitwiki GitWiki]
* [http://www.kernel.org/git/ kernel.org Gitweb interface] (Browse the Git and associated tools' repositories)
* [http://wiki.sourcemage.org/Git_Guide Straightforward Git guide] (for Git >= 1.5)

==Moodle Forum Discussions==

* [http://moodle.org/mod/forum/discuss.php?d=91998 Tracking Moodle CVS with git (take2)]
* [http://moodle.org/mod/forum/discuss.php?d=42275 Tracking Moodle CVS with git]
* [http://moodle.org/mod/forum/discuss.php?d=34472 Merging Custom Moodle Code With Stable Releases]
* [http://moodle.org/mod/forum/discuss.php?d=66412 Moving to git, need serious help]
* [http://moodle.org/mod/forum/discuss.php?d=99763 The continuing story of moving Moodle towards Git]

[[Category:Developer|Tracking Moodle CVS with git]]

Development:Tracking Moodle CVS with git

2008-10-01T22:22:07Z

Poltawski: Removed obsolete cogito docs + broken link

Note: Work in progress. These are notes for users that are comfortable with CVS and other SCMs, doing branching, merging, etc.

==Introduction==

If you plan on doing complicated or sustained development on Moodle, you will benefit from a local revision control system. Centralized systems like [http://www.nongnu.org/cvs/ CVS] and [http://subversion.tigris.org/ SVN] have limited capabilities for tracking vendor branches. For the ultimate experience you will want to use a system with distributed capabilities like [http://git.or.cz/ git]. Alternative tools for this include [http://www.selenic.com/mercurial/ Mercurial], [http://www.darcs.net/ Darcs] and [http://svk.elixus.org/ SVK].

GIT was developed by Linus Torvalds specifically for the Linux Kernel team. It is fast, fast, fast. Its usage is slightly different from CVS/SVN, but, if kernel developers can handle it ''Moodle developers will find it '''easy''' '' ;-)

Git is packaged for most major operating systems and an up to date list of packages can be found on [http://git.or.cz/ git.or.gz].

==Obtaining git==

You'll want:
* [http://www.kernel.org/pub/software/scm/git/ Git] (and its deps). It includes '''gitk''', a very good UI for visualising project history.
* [http://www.cobite.com/cvsps/ cvsps] (a dependency of git-cvsimport make sure you have the latest release)
* an additional git porcelain (frontend)
** [http://sourceforge.net/projects/qgit qgit] a nice GUI to view the project history, make commits, etc. Recommended! Note: qgit is tricky to compile by hand, get a .deb or an .rpm
** [http://www.procode.org/stgit/ stgit] (StackedGIT is mainly for users doing heavy cherry picking. Only recommended for very advanced SCM users.)

Git is still developing quickly, but good mature versions are already available in packaged format. This guide is written with GIT v1.4 and Cogito v0.17 in mind. Ubuntu and Debian Backports have up-to-date packages for GIT, Cogito and related packages. RPMs are usually also available via YUM. If you want to be on the bleeding/leading edge you can follow the ''maint'' or ''master'' branches of the GIT development code.

==Importing CVS==

To begin we must import the CVS repository into a local git filesystem using [http://www.kernel.org/pub/software/scm/git/docs/git-cvsimport.html git-cvsimport].

#!/bin/bash
CVSROOT=:pserver:anonymous@uk.cvs.moodle.org:/cvsroot/moodle
MODULE=moodle
INSTALLDIR=moodle
git-cvsimport -p x -v -k -o cvshead -d $CVSROOT -C $INSTALLDIR $MODULE &> cvsimport.log

(When might one want to use the -m -u and -s flags?)
-u purely cosmetic
-s slashes can cause problems but this doesn't seem to be the case with moodle
-m ?

This will run for a '''very''' long time as it checks out every revision of each file (including all branches). However, if you run the command after a successful initial run it will simply get the new revisions since it was last run. This creates a new git repository which can then be cloned and worked upon.

At the time of writing the size of a new Moodle git repository was approximately 866MB ''unpacked'', and 75MB packed. Newer versions of git pack the project during import, so your resulting repository plus checkout should be a bit over 100MB. Still, it is a good idea to pack the repository to make it smaller and faster, running

git-repack -a -d

With the -o cvshead flag, the "HEAD" of the CVS repo will appear as 'cvshead' in GIT, making things a lot clearer.

==Importing CVS Faster==

For initial imports, it is highly recommended that you fetch the CVS repository from one of Moodle [[CVS_for_Administrators#CVS_Servers | cvs servers]] and run the initial cvsimport against your local copy of the repository.

:: For the more adventurous: Keith Packard wrote another importer that does a better job at the initial import, called parsecvs. I use parsecvs for the initial import and then git-cvsimport daily for the incremental imports. --[[User:Martin Langhoff|Martin Langhoff]] 15:04, 19 July 2006 (WST)

:: It took me a few minutes to find the info in the Sourceforge docs on how to fetch the raw CVS repo via rsync. Just type 'rsync -av "rsync://moodle.cvs.sourceforge.net/cvsroot/moodle/moodle/*" .' (without the single quotes) and you'll be done --[[User:Iñaki Arenaza|Iñaki Arenaza]] 17:54, 25 December 2006 (CST).

:: This rsync command line doesn't work as-is anymore, since the repository was moved from sourceforge. Some mirrors (at least es.cvs.moodle.org at this time) provide rsync, but some don't (like eu.cvs.moodle.org). Find the best mirror for you and update the rsync call accordingly.

After the initial import, you can run git-cvsimport against your favorite cvs.moodle.org mirror repository, or update your local copy via rsync as sas demonstrated in [http://www.progsoc.org/~wildfire/git/update-repo.sh this script].

==Creating a Working Copy==

In order to use your repository you must clone yourself a working copy using [http://www.kernel.org/pub/software/scm/git/docs/git-clone.html git-clone].

(Let's assume your destination directory above was /pub/scm/moodle)

mkdir ~/git/moodle
cd ~/git/moodle
git clone /pub/scm/moodle.git

This will clone the git filesystem you created from the CVS import into a working copy that you may make changes to and commit to. You can then get updates from the main git repository that will be merged in amongst the changes to your local copy.

==Using git effectively with Moodle==

Git can be used to intelligently track changes for creating client installs based off your own local, base, customisations which in turn tracks changes to the main Moodle CVS.

If you have a local git copy of the Moodle CVS repository you will have a number of heads such as: '''MOODLE_15_STABLE''', '''MOODLE_16_STABLE''', and so on. If you wish to create a new branch for your local customisations, you can base it off an existing HEAD (one of the CVS branches that exist in the main Moodle repository). If you wish to create a new local branch, '''mymoodle''', based off the current stable 1.6 code you would:

cd /pub/scm/moodle.git
git branch mymoodle MOODLE_16_STABLE
git branch ''# lists all the current branches, should show '''mymoodle'''
git checkout mymoodle ''# you are now working on the '''mymoodle''' branch''
''# add some new files (i.e. themes, blocks, etc.)''
git add .
git status ''# Will list your uncommited files''
git commit -m "Added base customisations."

To return to the '''master''' HEAD, equivalent to the main Moodle CVS HEAD branch, you would have to use:

git branch master

If you were running a nightly script to pull new Moodle CVS updates into your main git repository via git-cvsimport, you should also merge any new changes into your local branch:

cd /pub/scm/moodle.git
git checkout mymoodle
git merge MOODLE_16_STABLE

If you wanted to use this custom local branch to create a new client install, you could use:

git-clone /pub/scm/moodle.git#mymoodle /home/someclient/public_html/
cd /home/someclient/public_html/
git checkout mymoodle origin/mymoodle

This would create a new git repository from your branch that could be used to do a client install which can, in turn, have the client's custom files added to it. This entire process can be automated so that your '''mymoodle''' branch is tracking upstream changes from the Moodle CVS ('''MOODLE_16_STABLE''' in this example) and your client installs are tracking changes against your local branch.

==Merging a patch from GIT into CVS==

If you have cvs access, you can use the [http://www.kernel.org/pub/software/scm/git/docs/git-cvsexportcommit.html git-cvsexportcommit] command to merge or cherry pick individual patches into ''upstream''.

::git-cvsexportcommit was initially written to do merges of NZOSVLE bugfixes into Moodle. Several tools and patches to GIT and Cogito come from the NZOSVLE team. [[User:Martin Langhoff|Martin Langhoff]] 14:55, 19 July 2006 (WST)

If the patches merge cleanly, future updates will recognise the already-applied patch. In that sense the merger in git is smarter than the merger in Cogito, consider using git-pull rather than cg-update. Note that if you see conflicts using git-pull you will have to work a bit harder to resolve them. Make sure you are familiar with handling merge conflicts with pure git before trying this.

==Preparing to merge large patch series or porting over to the next Moodle release==

When you have a large set of patches to apply to CVS, or a large set of custom patches to apply on top of the next release of Moodle, you can use [http://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html git-format-patch] to see what are your ''unmerged'' patches. git-format-patch tries hard to spot already-merged patches and skip them.

If you are applying patches to CVS, you can then use git-cvsexportcommit or plain old patch -p1 filename. If you are applying them to a new git branch (as you would to merge them on top of a new Moodle release) then you want to be on that new branch, and use git-am -3 -k filename.

If you are using git-am, make sure you learn how to deal with merge conflicts, and the use of git-update-index.

==Helpful Documentation==

* [http://www.kernel.org/pub/software/scm/git/docs/gitcvs-migration.html git for CVS users]
* [http://www.kernel.org/pub/software/scm/git/docs/git-cvsimport.html git-cvsimport man page]
* [http://www.kernel.org/pub/software/scm/git/docs/ git man pages]
* [http://vger.kernel.org/vger-lists.html The Git mailing list]
* [http://git.or.cz/gitwiki GitWiki]
* [http://www.kernel.org/git/ kernel.org Gitweb interface] (Browse the Git and associated tools' repositories)
* [http://wiki.sourcemage.org/Git_Guide Straightforward Git guide] (for Git >= 1.5)

==Moodle Forum Discussions==

* [http://moodle.org/mod/forum/discuss.php?d=91998 Tracking Moodle CVS with git (take2)]
* [http://moodle.org/mod/forum/discuss.php?d=42275 Tracking Moodle CVS with git]
* [http://moodle.org/mod/forum/discuss.php?d=34472 Merging Custom Moodle Code With Stable Releases]
* [http://moodle.org/mod/forum/discuss.php?d=66412 Moving to git, need serious help]
* [http://moodle.org/mod/forum/discuss.php?d=99763 The continuing story of moving Moodle towards Git]

[[Category:Developer|Tracking Moodle CVS with git]]

Development:Tracking Moodle CVS with git

2008-10-01T22:15:30Z

Poltawski: removed cogito commands - not sure about the new client install method

Note: Work in progress. These are notes for users that are comfortable with CVS and other SCMs, doing branching, merging, etc.

==Introduction==

If you plan on doing complicated or sustained development on Moodle, you will benefit from a local revision control system. Centralized systems like [http://www.nongnu.org/cvs/ CVS] and [http://subversion.tigris.org/ SVN] have limited capabilities for tracking vendor branches. For the ultimate experience you will want to use a system with distributed capabilities like [http://git.or.cz/ git]. Alternative tools for this include [http://www.selenic.com/mercurial/ Mercurial], [http://www.darcs.net/ Darcs] and [http://svk.elixus.org/ SVK].

GIT was developed by Linus Torvalds specifically for the Linux Kernel team. It is fast, fast, fast. Its usage is slightly different from CVS/SVN, but, if kernel developers can handle it ''Moodle developers will find it '''easy''' '' ;-)

Git is packaged for most major operating systems and an up to date list of packages can be found on [http://git.or.cz/ git.or.gz].

==Obtaining git==

You'll want:
* [http://www.kernel.org/pub/software/scm/git/ Git] (and its deps). It includes '''gitk''', a very good UI for visualising project history.
* [http://www.cobite.com/cvsps/ cvsps] (a dependency of git-cvsimport make sure you have the latest release)
* an additional git porcelain (frontend)
** [http://sourceforge.net/projects/qgit qgit] a nice GUI to view the project history, make commits, etc. Recommended! Note: qgit is tricky to compile by hand, get a .deb or an .rpm
** [http://www.procode.org/stgit/ stgit] (StackedGIT is mainly for users doing heavy cherry picking. Only recommended for very advanced SCM users.)

Git is still developing quickly, but good mature versions are already available in packaged format. This guide is written with GIT v1.4 and Cogito v0.17 in mind. Ubuntu and Debian Backports have up-to-date packages for GIT, Cogito and related packages. RPMs are usually also available via YUM. If you want to be on the bleeding/leading edge you can follow the ''maint'' or ''master'' branches of the GIT development code.

==Importing CVS==

To begin we must import the CVS repository into a local git filesystem using [http://www.kernel.org/pub/software/scm/git/docs/git-cvsimport.html git-cvsimport].

#!/bin/bash
CVSROOT=:pserver:anonymous@uk.cvs.moodle.org:/cvsroot/moodle
MODULE=moodle
INSTALLDIR=moodle
git-cvsimport -p x -v -k -o cvshead -d $CVSROOT -C $INSTALLDIR $MODULE &> cvsimport.log

(When might one want to use the -m -u and -s flags?)
-u purely cosmetic
-s slashes can cause problems but this doesn't seem to be the case with moodle
-m ?

This will run for a '''very''' long time as it checks out every revision of each file (including all branches). However, if you run the command after a successful initial run it will simply get the new revisions since it was last run. This creates a new git repository which can then be cloned and worked upon.

At the time of writing the size of a new Moodle git repository was approximately 866MB ''unpacked'', and 75MB packed. Newer versions of git pack the project during import, so your resulting repository plus checkout should be a bit over 100MB. Still, it is a good idea to pack the repository to make it smaller and faster, running

git-repack -a -d

With the -o cvshead flag, the "HEAD" of the CVS repo will appear as 'cvshead' in GIT, making things a lot clearer.

==Importing CVS Faster==

For initial imports, it is highly recommended that you fetch the CVS repository from one of Moodle [[CVS_for_Administrators#CVS_Servers | cvs servers]] and run the initial cvsimport against your local copy of the repository.

:: For the more adventurous: Keith Packard wrote another importer that does a better job at the initial import, called parsecvs. I use parsecvs for the initial import and then git-cvsimport daily for the incremental imports. --[[User:Martin Langhoff|Martin Langhoff]] 15:04, 19 July 2006 (WST)

:: It took me a few minutes to find the info in the Sourceforge docs on how to fetch the raw CVS repo via rsync. Just type 'rsync -av "rsync://moodle.cvs.sourceforge.net/cvsroot/moodle/moodle/*" .' (without the single quotes) and you'll be done --[[User:Iñaki Arenaza|Iñaki Arenaza]] 17:54, 25 December 2006 (CST).

:: This rsync command line doesn't work as-is anymore, since the repository was moved from sourceforge. Some mirrors (at least es.cvs.moodle.org at this time) provide rsync, but some don't (like eu.cvs.moodle.org). Find the best mirror for you and update the rsync call accordingly.

After the initial import, you can run git-cvsimport against your favorite cvs.moodle.org mirror repository, or update your local copy via rsync as sas demonstrated in [http://www.progsoc.org/~wildfire/git/update-repo.sh this script].

==Creating a Working Copy==

In order to use your repository you must clone yourself a working copy using [http://www.kernel.org/pub/software/scm/git/docs/git-clone.html git-clone].

(Let's assume your destination directory above was /pub/scm/moodle)

mkdir ~/git/moodle
cd ~/git/moodle
git clone /pub/scm/moodle.git

This will clone the git filesystem you created from the CVS import into a working copy that you may make changes to and commit to. You can then get updates from the main git repository that will be merged in amongst the changes to your local copy.

==Using git effectively with Moodle==

Git can be used to intelligently track changes for creating client installs based off your own local, base, customisations which in turn tracks changes to the main Moodle CVS.

If you have a local git copy of the Moodle CVS repository you will have a number of heads such as: '''MOODLE_15_STABLE''', '''MOODLE_16_STABLE''', and so on. If you wish to create a new branch for your local customisations, you can base it off an existing HEAD (one of the CVS branches that exist in the main Moodle repository). If you wish to create a new local branch, '''mymoodle''', based off the current stable 1.6 code you would:

cd /pub/scm/moodle.git
git branch mymoodle MOODLE_16_STABLE
git branch ''# lists all the current branches, should show '''mymoodle'''
git checkout mymoodle ''# you are now working on the '''mymoodle''' branch''
''# add some new files (i.e. themes, blocks, etc.)''
git add .
git status ''# Will list your uncommited files''
git commit -m "Added base customisations."

To return to the '''master''' HEAD, equivalent to the main Moodle CVS HEAD branch, you would have to use:

git branch master

If you were running a nightly script to pull new Moodle CVS updates into your main git repository via git-cvsimport, you should also merge any new changes into your local branch:

cd /pub/scm/moodle.git
git checkout mymoodle
git merge MOODLE_16_STABLE

If you wanted to use this custom local branch to create a new client install, you could use:

git-clone /pub/scm/moodle.git#mymoodle /home/someclient/public_html/
cd /home/someclient/public_html/
git checkout mymoodle origin/mymoodle

This would create a new git repository from your branch that could be used to do a client install which can, in turn, have the client's custom files added to it. This entire process can be automated so that your '''mymoodle''' branch is tracking upstream changes from the Moodle CVS ('''MOODLE_16_STABLE''' in this example) and your client installs are tracking changes against your local branch.

==Merging a patch from GIT into CVS==

If you have cvs access, you can use the [http://www.kernel.org/pub/software/scm/git/docs/git-cvsexportcommit.html git-cvsexportcommit] command to merge or cherry pick individual patches into ''upstream''.

::git-cvsexportcommit was initially written to do merges of NZOSVLE bugfixes into Moodle. Several tools and patches to GIT and Cogito come from the NZOSVLE team. [[User:Martin Langhoff|Martin Langhoff]] 14:55, 19 July 2006 (WST)

If the patches merge cleanly, future updates will recognise the already-applied patch. In that sense the merger in git is smarter than the merger in Cogito, consider using git-pull rather than cg-update. Note that if you see conflicts using git-pull you will have to work a bit harder to resolve them. Make sure you are familiar with handling merge conflicts with pure git before trying this.

==Preparing to merge large patch series or porting over to the next Moodle release==

When you have a large set of patches to apply to CVS, or a large set of custom patches to apply on top of the next release of Moodle, you can use [http://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html git-format-patch] to see what are your ''unmerged'' patches. git-format-patch tries hard to spot already-merged patches and skip them.

If you are applying patches to CVS, you can then use git-cvsexportcommit or plain old patch -p1 filename. If you are applying them to a new git branch (as you would to merge them on top of a new Moodle release) then you want to be on that new branch, and use git-am -3 -k filename.

If you are using git-am, make sure you learn how to deal with merge conflicts, and the use of git-update-index.

==Helpful Documentation==

* [http://wellington.pm.org/archive/200510/git/ cogito tutorial by martin langhoff]
* [http://www.kernel.org/pub/software/scm/git/docs/tutorial.html git tutorial]
* [http://www.kernel.org/pub/software/scm/git/docs/gitcvs-migration.html git for CVS users]
* [http://www.kernel.org/pub/software/scm/git/docs/git-cvsimport.html git-cvsimport man page]
* [http://www.kernel.org/pub/software/scm/cogito/docs/ cogito man pages]
* [http://www.kernel.org/pub/software/scm/git/docs/ git man pages]
* [http://vger.kernel.org/vger-lists.html The Git mailing list]
* [http://git.or.cz/gitwiki GitWiki]
* [http://www.kernel.org/git/ kernel.org Gitweb interface] (Browse the Git and associated tools' repositories)
* [http://wiki.sourcemage.org/Git_Guide Straightforward Git guide] (for Git >= 1.5)

==Moodle Forum Discussions==

* [http://moodle.org/mod/forum/discuss.php?d=91998 Tracking Moodle CVS with git (take2)]
* [http://moodle.org/mod/forum/discuss.php?d=42275 Tracking Moodle CVS with git]
* [http://moodle.org/mod/forum/discuss.php?d=34472 Merging Custom Moodle Code With Stable Releases]
* [http://moodle.org/mod/forum/discuss.php?d=66412 Moving to git, need serious help]
* [http://moodle.org/mod/forum/discuss.php?d=99763 The continuing story of moving Moodle towards Git]

[[Category:Developer|Tracking Moodle CVS with git]]

Development talk:Tracking Moodle CVS with git

2008-10-01T22:04:13Z

Poltawski: New page: I am currently doing a bit of an overhaul of this page to remove obsolete coigto stuff - dont really see a reason why anyone would use it or want docs for it.--~~~~

I am currently doing a bit of an overhaul of this page to remove obsolete coigto stuff - dont really see a reason why anyone would use it or want docs for it.--[[User:Dan Poltawski|Dan Poltawski]] 17:04, 1 October 2008 (CDT)

Development:Tracking Moodle CVS with git

2008-10-01T22:03:14Z

Poltawski: removed cogito commands - cogito is obsolete, and the git interface is quite friendly now

Note: Work in progress. These are notes for users that are comfortable with CVS and other SCMs, doing branching, merging, etc.

==Introduction==

If you plan on doing complicated or sustained development on Moodle, you will benefit from a local revision control system. Centralized systems like [http://www.nongnu.org/cvs/ CVS] and [http://subversion.tigris.org/ SVN] have limited capabilities for tracking vendor branches. For the ultimate experience you will want to use a system with distributed capabilities like [http://git.or.cz/ git]. Alternative tools for this include [http://www.selenic.com/mercurial/ Mercurial], [http://www.darcs.net/ Darcs] and [http://svk.elixus.org/ SVK].

GIT was developed by Linus Torvalds specifically for the Linux Kernel team. It is fast, fast, fast. Its usage is slightly different from CVS/SVN, but, if kernel developers can handle it ''Moodle developers will find it '''easy''' '' ;-)

Git is packaged for most major operating systems and an up to date list of packages can be found on [http://git.or.cz/ git.or.gz].

==Obtaining git==

You'll want:
* [http://www.kernel.org/pub/software/scm/git/ Git] (and its deps). It includes '''gitk''', a very good UI for visualising project history.
* [http://www.cobite.com/cvsps/ cvsps] (a dependency of git-cvsimport make sure you have the latest release)
* an additional git porcelain (frontend)
** [http://sourceforge.net/projects/qgit qgit] a nice GUI to view the project history, make commits, etc. Recommended! Note: qgit is tricky to compile by hand, get a .deb or an .rpm
** [http://www.procode.org/stgit/ stgit] (StackedGIT is mainly for users doing heavy cherry picking. Only recommended for very advanced SCM users.)

Git is still developing quickly, but good mature versions are already available in packaged format. This guide is written with GIT v1.4 and Cogito v0.17 in mind. Ubuntu and Debian Backports have up-to-date packages for GIT, Cogito and related packages. RPMs are usually also available via YUM. If you want to be on the bleeding/leading edge you can follow the ''maint'' or ''master'' branches of the GIT development code.

==Importing CVS==

To begin we must import the CVS repository into a local git filesystem using [http://www.kernel.org/pub/software/scm/git/docs/git-cvsimport.html git-cvsimport].

#!/bin/bash
CVSROOT=:pserver:anonymous@uk.cvs.moodle.org:/cvsroot/moodle
MODULE=moodle
INSTALLDIR=moodle
git-cvsimport -p x -v -k -o cvshead -d $CVSROOT -C $INSTALLDIR $MODULE &> cvsimport.log

(When might one want to use the -m -u and -s flags?)
-u purely cosmetic
-s slashes can cause problems but this doesn't seem to be the case with moodle
-m ?

This will run for a '''very''' long time as it checks out every revision of each file (including all branches). However, if you run the command after a successful initial run it will simply get the new revisions since it was last run. This creates a new git repository which can then be cloned and worked upon.

At the time of writing the size of a new Moodle git repository was approximately 866MB ''unpacked'', and 75MB packed. Newer versions of git pack the project during import, so your resulting repository plus checkout should be a bit over 100MB. Still, it is a good idea to pack the repository to make it smaller and faster, running

git-repack -a -d

With the -o cvshead flag, the "HEAD" of the CVS repo will appear as 'cvshead' in GIT, making things a lot clearer.

==Importing CVS Faster==

For initial imports, it is highly recommended that you fetch the CVS repository from one of Moodle [[CVS_for_Administrators#CVS_Servers | cvs servers]] and run the initial cvsimport against your local copy of the repository.

:: For the more adventurous: Keith Packard wrote another importer that does a better job at the initial import, called parsecvs. I use parsecvs for the initial import and then git-cvsimport daily for the incremental imports. --[[User:Martin Langhoff|Martin Langhoff]] 15:04, 19 July 2006 (WST)

:: It took me a few minutes to find the info in the Sourceforge docs on how to fetch the raw CVS repo via rsync. Just type 'rsync -av "rsync://moodle.cvs.sourceforge.net/cvsroot/moodle/moodle/*" .' (without the single quotes) and you'll be done --[[User:Iñaki Arenaza|Iñaki Arenaza]] 17:54, 25 December 2006 (CST).

:: This rsync command line doesn't work as-is anymore, since the repository was moved from sourceforge. Some mirrors (at least es.cvs.moodle.org at this time) provide rsync, but some don't (like eu.cvs.moodle.org). Find the best mirror for you and update the rsync call accordingly.

After the initial import, you can run git-cvsimport against your favorite cvs.moodle.org mirror repository, or update your local copy via rsync as sas demonstrated in [http://www.progsoc.org/~wildfire/git/update-repo.sh this script].

==Creating a Working Copy==

In order to use your repository you must clone yourself a working copy using [http://www.kernel.org/pub/software/scm/git/docs/git-clone.html git-clone].

(Let's assume your destination directory above was /pub/scm/moodle)

mkdir ~/git/moodle
cd ~/git/moodle
git clone /pub/scm/moodle.git

This will clone the git filesystem you created from the CVS import into a working copy that you may make changes to and commit to. You can then get updates from the main git repository that will be merged in amongst the changes to your local copy.

==Using git effectively with Moodle==

Git can be used to intelligently track changes for creating client installs based off your own local, base, customisations which in turn tracks changes to the main Moodle CVS.

If you have a local git copy of the Moodle CVS repository you will have a number of heads such as: '''MOODLE_15_STABLE''', '''MOODLE_16_STABLE''', and so on. If you wish to create a new branch for your local customisations, you can base it off an existing HEAD (one of the CVS branches that exist in the main Moodle repository). If you wish to create a new local branch, '''mymoodle''', based off the current stable 1.6 code you would:

cd /pub/scm/moodle.git
git branch mymoodle MOODLE_16_STABLE
git branch ''# lists all the current branches, should show '''mymoodle'''
git checkout mymoodle ''# you are now working on the '''mymoodle''' branch''
''# add some new files (i.e. themes, blocks, etc.)''
cg-add -r .
cg-status ''# Will list your uncommited files''
cg-commit -m "Added base customisations."

To return to the '''master''' HEAD, equivalent to the main Moodle CVS HEAD branch, you would have to use:

git branch master

If you were running a nightly script to pull new Moodle CVS updates into your main git repository via git-cvsimport, you should also merge any new changes into your local branch:

cd /pub/scm/moodle.git
git checkout mymoodle
cg-merge MOODLE_16_STABLE

If you wanted to use this custom local branch to create a new client install, you would use:

cg-clone /pub/scm/moodle.git#mymoodle /home/someclient/public_html/

This would create a new git repository from your branch that could be used to do a client install which can, in turn, have the client's custom files added to it. This entire process can be automated so that your '''mymoodle''' branch is tracking upstream changes from the Moodle CVS ('''MOODLE_16_STABLE''' in this example) and your client installs are tracking changes against your local branch.

==Merging a patch from GIT into CVS==

If you have cvs access, you can use the [http://www.kernel.org/pub/software/scm/git/docs/git-cvsexportcommit.html git-cvsexportcommit] command to merge or cherry pick individual patches into ''upstream''.

::git-cvsexportcommit was initially written to do merges of NZOSVLE bugfixes into Moodle. Several tools and patches to GIT and Cogito come from the NZOSVLE team. [[User:Martin Langhoff|Martin Langhoff]] 14:55, 19 July 2006 (WST)

If the patches merge cleanly, future updates will recognise the already-applied patch. In that sense the merger in git is smarter than the merger in Cogito, consider using git-pull rather than cg-update. Note that if you see conflicts using git-pull you will have to work a bit harder to resolve them. Make sure you are familiar with handling merge conflicts with pure git before trying this.

==Preparing to merge large patch series or porting over to the next Moodle release==

When you have a large set of patches to apply to CVS, or a large set of custom patches to apply on top of the next release of Moodle, you can use [http://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html git-format-patch] to see what are your ''unmerged'' patches. git-format-patch tries hard to spot already-merged patches and skip them.

If you are applying patches to CVS, you can then use git-cvsexportcommit or plain old patch -p1 filename. If you are applying them to a new git branch (as you would to merge them on top of a new Moodle release) then you want to be on that new branch, and use git-am -3 -k filename.

If you are using git-am, make sure you learn how to deal with merge conflicts, and the use of git-update-index.

==Helpful Documentation==

* [http://wellington.pm.org/archive/200510/git/ cogito tutorial by martin langhoff]
* [http://www.kernel.org/pub/software/scm/git/docs/tutorial.html git tutorial]
* [http://www.kernel.org/pub/software/scm/git/docs/gitcvs-migration.html git for CVS users]
* [http://www.kernel.org/pub/software/scm/git/docs/git-cvsimport.html git-cvsimport man page]
* [http://www.kernel.org/pub/software/scm/cogito/docs/ cogito man pages]
* [http://www.kernel.org/pub/software/scm/git/docs/ git man pages]
* [http://vger.kernel.org/vger-lists.html The Git mailing list]
* [http://git.or.cz/gitwiki GitWiki]
* [http://www.kernel.org/git/ kernel.org Gitweb interface] (Browse the Git and associated tools' repositories)
* [http://wiki.sourcemage.org/Git_Guide Straightforward Git guide] (for Git >= 1.5)

==Moodle Forum Discussions==

* [http://moodle.org/mod/forum/discuss.php?d=91998 Tracking Moodle CVS with git (take2)]
* [http://moodle.org/mod/forum/discuss.php?d=42275 Tracking Moodle CVS with git]
* [http://moodle.org/mod/forum/discuss.php?d=34472 Merging Custom Moodle Code With Stable Releases]
* [http://moodle.org/mod/forum/discuss.php?d=66412 Moving to git, need serious help]
* [http://moodle.org/mod/forum/discuss.php?d=99763 The continuing story of moving Moodle towards Git]

[[Category:Developer|Tracking Moodle CVS with git]]

Development:Tracking Moodle CVS with git

2008-10-01T21:58:59Z

Poltawski: removed obsolete information about windows - replaced with note about packaged binaries

Note: Work in progress. These are notes for users that are comfortable with CVS and other SCMs, doing branching, merging, etc.

==Introduction==

If you plan on doing complicated or sustained development on Moodle, you will benefit from a local revision control system. Centralized systems like [http://www.nongnu.org/cvs/ CVS] and [http://subversion.tigris.org/ SVN] have limited capabilities for tracking vendor branches. For the ultimate experience you will want to use a system with distributed capabilities like [http://git.or.cz/ git]. Alternative tools for this include [http://www.selenic.com/mercurial/ Mercurial], [http://www.darcs.net/ Darcs] and [http://svk.elixus.org/ SVK].

GIT was developed by Linus Torvalds specifically for the Linux Kernel team. It is fast, fast, fast. Its usage is slightly different from CVS/SVN, but, if kernel developers can handle it ''Moodle developers will find it '''easy''' '' ;-)

Git is packaged for most major operating systems and an up to date list of packages can be found on [http://git.or.cz/ git.or.gz].

==Obtaining git==

You'll want:
* [http://www.kernel.org/pub/software/scm/git/ Git] (and its deps). It includes '''gitk''', a very good UI for visualising project history.
* [http://www.cobite.com/cvsps/ cvsps] (a dependency of git-cvsimport make sure you have the latest release)
* an additional git porcelain (frontend)
** [http://sourceforge.net/projects/qgit qgit] a nice GUI to view the project history, make commits, etc. Recommended! Note: qgit is tricky to compile by hand, get a .deb or an .rpm
** [http://www.procode.org/stgit/ stgit] (StackedGIT is mainly for users doing heavy cherry picking. Only recommended for very advanced SCM users.)

Git is still developing quickly, but good mature versions are already available in packaged format. This guide is written with GIT v1.4 and Cogito v0.17 in mind. Ubuntu and Debian Backports have up-to-date packages for GIT, Cogito and related packages. RPMs are usually also available via YUM. If you want to be on the bleeding/leading edge you can follow the ''maint'' or ''master'' branches of the GIT development code.

==Importing CVS==

To begin we must import the CVS repository into a local git filesystem using [http://www.kernel.org/pub/software/scm/git/docs/git-cvsimport.html git-cvsimport].

#!/bin/bash
CVSROOT=:pserver:anonymous@uk.cvs.moodle.org:/cvsroot/moodle
MODULE=moodle
INSTALLDIR=moodle
git-cvsimport -p x -v -k -o cvshead -d $CVSROOT -C $INSTALLDIR $MODULE &> cvsimport.log

(When might one want to use the -m -u and -s flags?)
-u purely cosmetic
-s slashes can cause problems but this doesn't seem to be the case with moodle
-m ?

This will run for a '''very''' long time as it checks out every revision of each file (including all branches). However, if you run the command after a successful initial run it will simply get the new revisions since it was last run. This creates a new git repository which can then be cloned and worked upon.

At the time of writing the size of a new Moodle git repository was approximately 866MB ''unpacked'', and 75MB packed. Newer versions of git pack the project during import, so your resulting repository plus checkout should be a bit over 100MB. Still, it is a good idea to pack the repository to make it smaller and faster, running

git-repack -a -d

With the -o cvshead flag, the "HEAD" of the CVS repo will appear as 'cvshead' in GIT, making things a lot clearer.

==Importing CVS Faster==

For initial imports, it is highly recommended that you fetch the CVS repository from one of Moodle [[CVS_for_Administrators#CVS_Servers | cvs servers]] and run the initial cvsimport against your local copy of the repository.

:: For the more adventurous: Keith Packard wrote another importer that does a better job at the initial import, called parsecvs. I use parsecvs for the initial import and then git-cvsimport daily for the incremental imports. --[[User:Martin Langhoff|Martin Langhoff]] 15:04, 19 July 2006 (WST)

:: It took me a few minutes to find the info in the Sourceforge docs on how to fetch the raw CVS repo via rsync. Just type 'rsync -av "rsync://moodle.cvs.sourceforge.net/cvsroot/moodle/moodle/*" .' (without the single quotes) and you'll be done --[[User:Iñaki Arenaza|Iñaki Arenaza]] 17:54, 25 December 2006 (CST).

:: This rsync command line doesn't work as-is anymore, since the repository was moved from sourceforge. Some mirrors (at least es.cvs.moodle.org at this time) provide rsync, but some don't (like eu.cvs.moodle.org). Find the best mirror for you and update the rsync call accordingly.

After the initial import, you can run git-cvsimport against your favorite cvs.moodle.org mirror repository, or update your local copy via rsync as sas demonstrated in [http://www.progsoc.org/~wildfire/git/update-repo.sh this script].

==Creating a Working Copy==

In order to use your repository you must clone yourself a working copy using [http://www.kernel.org/pub/software/scm/git/docs/git-clone.html git-clone].

(Let's assume your destination directory above was /pub/scm/moodle)

mkdir ~/git/moodle
cd ~/git/moodle
git clone /pub/scm/moodle.git

This will clone the git filesystem you created from the CVS import into a working copy that you may make changes to and commit to. You can then get updates from the main git repository that will be merged in amongst the changes to your local copy.

If you are using the Cogito tools, you can use [http://www.kernel.org/pub/software/scm/cogito/docs/cg-clone.html cg-clone] to do this instead:

cg-clone /pub/scm/moodle.git ~/git/moodle

==Using git effectively with Moodle==

Git can be used to intelligently track changes for creating client installs based off your own local, base, customisations which in turn tracks changes to the main Moodle CVS.

If you have a local git copy of the Moodle CVS repository you will have a number of heads such as: '''MOODLE_15_STABLE''', '''MOODLE_16_STABLE''', and so on. If you wish to create a new branch for your local customisations, you can base it off an existing HEAD (one of the CVS branches that exist in the main Moodle repository). If you wish to create a new local branch, '''mymoodle''', based off the current stable 1.6 code you would:

cd /pub/scm/moodle.git
git branch mymoodle MOODLE_16_STABLE
git branch ''# lists all the current branches, should show '''mymoodle'''
git checkout mymoodle ''# you are now working on the '''mymoodle''' branch''
''# add some new files (i.e. themes, blocks, etc.)''
cg-add -r .
cg-status ''# Will list your uncommited files''
cg-commit -m "Added base customisations."

To return to the '''master''' HEAD, equivalent to the main Moodle CVS HEAD branch, you would have to use:

git branch master

If you were running a nightly script to pull new Moodle CVS updates into your main git repository via git-cvsimport, you should also merge any new changes into your local branch:

cd /pub/scm/moodle.git
git checkout mymoodle
cg-merge MOODLE_16_STABLE

If you wanted to use this custom local branch to create a new client install, you would use:

cg-clone /pub/scm/moodle.git#mymoodle /home/someclient/public_html/

This would create a new git repository from your branch that could be used to do a client install which can, in turn, have the client's custom files added to it. This entire process can be automated so that your '''mymoodle''' branch is tracking upstream changes from the Moodle CVS ('''MOODLE_16_STABLE''' in this example) and your client installs are tracking changes against your local branch.

==Merging a patch from GIT into CVS==

If you have cvs access, you can use the [http://www.kernel.org/pub/software/scm/git/docs/git-cvsexportcommit.html git-cvsexportcommit] command to merge or cherry pick individual patches into ''upstream''.

::git-cvsexportcommit was initially written to do merges of NZOSVLE bugfixes into Moodle. Several tools and patches to GIT and Cogito come from the NZOSVLE team. [[User:Martin Langhoff|Martin Langhoff]] 14:55, 19 July 2006 (WST)

If the patches merge cleanly, future updates will recognise the already-applied patch. In that sense the merger in git is smarter than the merger in Cogito, consider using git-pull rather than cg-update. Note that if you see conflicts using git-pull you will have to work a bit harder to resolve them. Make sure you are familiar with handling merge conflicts with pure git before trying this.

==Preparing to merge large patch series or porting over to the next Moodle release==

When you have a large set of patches to apply to CVS, or a large set of custom patches to apply on top of the next release of Moodle, you can use [http://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html git-format-patch] to see what are your ''unmerged'' patches. git-format-patch tries hard to spot already-merged patches and skip them.

If you are applying patches to CVS, you can then use git-cvsexportcommit or plain old patch -p1 filename. If you are applying them to a new git branch (as you would to merge them on top of a new Moodle release) then you want to be on that new branch, and use git-am -3 -k filename.

If you are using git-am, make sure you learn how to deal with merge conflicts, and the use of git-update-index.

==Helpful Documentation==

* [http://wellington.pm.org/archive/200510/git/ cogito tutorial by martin langhoff]
* [http://www.kernel.org/pub/software/scm/git/docs/tutorial.html git tutorial]
* [http://www.kernel.org/pub/software/scm/git/docs/gitcvs-migration.html git for CVS users]
* [http://www.kernel.org/pub/software/scm/git/docs/git-cvsimport.html git-cvsimport man page]
* [http://www.kernel.org/pub/software/scm/cogito/docs/ cogito man pages]
* [http://www.kernel.org/pub/software/scm/git/docs/ git man pages]
* [http://vger.kernel.org/vger-lists.html The Git mailing list]
* [http://git.or.cz/gitwiki GitWiki]
* [http://www.kernel.org/git/ kernel.org Gitweb interface] (Browse the Git and associated tools' repositories)
* [http://wiki.sourcemage.org/Git_Guide Straightforward Git guide] (for Git >= 1.5)

==Moodle Forum Discussions==

* [http://moodle.org/mod/forum/discuss.php?d=91998 Tracking Moodle CVS with git (take2)]
* [http://moodle.org/mod/forum/discuss.php?d=42275 Tracking Moodle CVS with git]
* [http://moodle.org/mod/forum/discuss.php?d=34472 Merging Custom Moodle Code With Stable Releases]
* [http://moodle.org/mod/forum/discuss.php?d=66412 Moving to git, need serious help]
* [http://moodle.org/mod/forum/discuss.php?d=99763 The continuing story of moving Moodle towards Git]

[[Category:Developer|Tracking Moodle CVS with git]]

CVS for Administrators

2008-09-21T20:35:59Z

Poltawski: updated luns website

The [http://cvs.moodle.org/ CVS archive] contains all the source code for Moodle. You can use a CVS program to extract versions ranging from the most stable release to the most cutting-edge development version. CVS can be an extremely convenient way of maintaining a Moodle server.

[[Image:Cvstree.png|CVS tree]]

Developers may have selective write access to the Moodle CVS archive (see [[CVS for Developers]] for details about how to do this). However, most people only need read-only access, so they can just connect to one of the mirrors using '''anonymous CVS''' as described below. There can however currently be a delay of up to 1 hour between the time a developer commits changes to developer CVS and the time it becomes available on anonymous CVS.

==CVS Servers==

Please choose the closest CVS mirror server to you from this list:

{| border="1" cellpadding="4" cellspacing="0"
|'''Country'''
|'''Server'''
|'''Provided by'''
|-
|EU
|'''eu.cvs.moodle.org'''
|[http://www.open.ac.uk/ The Open University] In case of trouble, contact [mailto:r.t.c.norfor@open.ac.uk Rod Norfor] or [mailto:d.a.woolhead@open.ac.uk Derek Woolhead]
|-
|ES
|'''es.cvs.moodle.org'''
|[http://www.mondragon.edu/ Mondragon Unibertsitatea] In case of trouble, contact [mailto:iarenuno@eteo.mondragon.edu iarenuno@eteo.mondragon.edu] or [mailto:iarenaza@escomposlinux.org iarenaza@escomposlinux.org]
|-
|UK
|'''uk.cvs.moodle.org'''
|Cumbria and Lancashire Education Online ([http://www.cleo.net.uk/ CLEO]) in collaboration with Lancaster University Network Services ([http://www.luns.net.uk/ LUNS])
|-
|US
|'''us.cvs.moodle.org'''
|San Francisco State University, Academic Technology ([http://www.sfsu.edu/ SFSU]). In case of trouble, contact [mailto:ilearn@sfsu.edu iLearn support]
|-
|US
|'''us2.cvs.moodle.org'''
|[http://www.contractorsinstitute.com The Contractors Institute] pserver and viewvc. In case of trouble contact [mailto:cvs@contractorsinstitute.com network support].
|}

Replace the SERVER.cvs.moodle.org in the instructions below with the server you chose above!

For up-to-date alerts about planned or unplanned outages on any of these servers subscribe to the [http://lists.moodle.org/info/outages Moodle Outage mailing list].

(If you would like to contribute to the project by running a mirror, please see [[How to set up a CVS mirror]])

===Switching to a new server===

If you were ''already'' using CVS and want to switch to a different server, you'll probably need to make a small change so that the control files in your working copy will point to the new mirrors.

====Switching to a new server on Unix====

Use a shell command like this to change existing installations to point to the new mirror (UK mirror used in this example):

'''find . -type f -name Root -print0 | xargs -0 perl -pi -e 's/\@moodle\.cvs\.sourceforge\.net/\@uk\.cvs\.moodle\.org/''''

It can be run from /home, say, to fix multiple sites at once.

====Switching to a new server with TortoiseCVS====

If you were ''already'' using Tortoise CVS on Windows it's tricky, because Tortoise doesn't have any interface for changing the server. http://www.tortoisecvs.org/faq.html#changecvsroot explains it. But basically,

1) Install WinCVS and launch it.
2) Navigate to and select your Moodle folder.
3) Choose 'Macros'->CVS->Change Root from the menu.
4) Accept (or change) the default for the 'old' server.
5) Type the new server name. OK!

It takes a few seconds to go through all of the '''cvs''' folders and update the '''root''' files.

By the way, if you don't want to install WinCVS, another way of doing this is to uninstall your TortoiseCVS client on Windows, then do a regedit to clean up all the tortoisecvs related entries (might not be necessary), then reinstall TortoiseCVS client again (a good reason to upgrade to the most recent version of TortoiseCVS!). I have tested this and it cleared up the original setting of the original anonymous CVS server setting.

==Instructions==

===From a Unix computer===

To connect and login for the first time to the CVS server, you can use this command (remember to replace '''SERVER.cvs.moodle.org''' in the instructions below with the mirror server you chose above):

cvs -d:pserver:anonymous@SERVER.cvs.moodle.org:/cvsroot/moodle login

There is no password - when asked for one, just hit Enter.

To checkout (download) the entire Moodle code for the first time, use this command to get the latest WEEKLY version (generally the latest, most bug free version):

cvs -z3 -d:pserver:anonymous@SERVER.cvs.moodle.org:/cvsroot/moodle co -P -r MOODLE_19_WEEKLY moodle

Or the latest development version (not for production use):

cvs -z3 -d:pserver:anonymous@SERVER.cvs.moodle.org:/cvsroot/moodle co -P moodle

Or the modules in Contrib

cvs -z3 -d:pserver:anonymous@SERVER.cvs.moodle.org:/cvsroot/moodle co contrib

Later, to update your local copy of Moodle to the current version in CVS you just need to go into your local Moodle directory and type:

cvs update -dP

To update your local copy of Moodle to a new version (e.g. from 1.8+ to 1.9), go into your local Moodle directory and type:

cvs update -dP -r MOODLE_19_STABLE

To update your local copy and to save the log of the process, use the following command instead the previous one:

cvs update -dP -r MOODLE_19_STABLE | tee upgrade.log

Then look at the upgrade.log, notably look for lines starting with "C" (conflict):

grep '^C' upgrade.log

Conflicts may appear in case you have manually modified your source files. You have to resolve conflicts before using the site. See [[CVS for Developers]] for more details.

===Changing the directory name===

By default, the CVS checkout creates a diectory on your webserver called 'moodle'. If you want your Moodle installation in a different directory, you can change the name of the directory that it will checkout the files to, by typing the follwing. This would download the MOODLE_19_STABLE branch into a directory called "mydirectory" (-d mydirectory).

cvs -z3 -d:pserver:anonymous@SERVER.cvs.moodle.org:/cvsroot/moodle co '''-d mydirectory''' -r MOODLE_19_STABLE moodle

Or the latest development version to a directory called 'moodle-dev':

cvs -z3 -d:pserver:anonymous@SERVER.cvs.moodle.org:/cvsroot/moodle co '''-d moodle-dev''' moodle

You can also change the name of the directory after the files are downloaded, and before you go through the Moodle install process. If you change the name of the directory before install, it will not affect anything during the install or during a CVS update. If you change the name of the directory after an install, you will need to change the config.php to reflect the name change ([[Moodle_migration#Migrating_a_complete_Moodle_site|guidance here]]). It won't affect the CVS update though.

===Change directory owner===

Depending on your webserver setup, you may well need to change the owner of the directory to the webserver user. Follow this step if you get permissions error when you try to access the page. For apache:

chown -R www-data:www-data moodle

===From a Windows computer===

To get started with a fresh copy of Moodle, follow the following steps (remember to replace '''SERVER.cvs.moodle.org''' in the instructions below with the mirror server you chose above):[[Image:CVS moodle settings for tortoise CVS.jpg|thumb|Tortoise CVS Screen capture]]
[[Image:Ecran cvs.jpg|thumb|Tortoise CVS (real name) Screen capture]]
# Get TortoiseCVS from [http://www.tortoisecvs.org/ tortoisecvs.org] and install it, then reboot.
# Find or create a new folder somewhere where you want Moodle to be downloaded to.
# Right-mouse-click that folder and choose "CVS Checkout" from the menu. You should see a dialog box.
# Copy this text into the CVSROOT field: <code>:pserver:anonymous@SERVER.cvs.moodle.org:/cvsroot/moodle</code>
#* NOTE - replace "SERVER" with "eu", "es", "uk" or "us" depending on your location.
# Under the "Module" field, type "moodle" to get moodle. (Other options here include"contrib" to get the contrib directory of hacks and addons, or "mysql" to get the optional MySQL Admin module).
#* For the latest STABLE version, click on the "Revision" tab and then check the radio button labelled "Choose branch or tag". From the drop-down menu select MOODLE_18_STABLE.
#* If you don't see the very latest version in the long drop-down list under Branch or tag name, click the Update List button next to it and wait for the list to be updated.
#* For the latest UNSTABLE development version, the radio-button "Use HEAD branch" in the Revision tab should be checked.
# Press the button: "OK" and everything should be downloaded.

Later, to update your local copy of Moodle to the current version in CVS, just right-mouse-click the folder and choose "CVS Update".

Note that the enclosing moodle folder is self-contained - you can move it anywhere you like or even rename it.

'''NOTE.-''' Admins with a developer account on cvs.moodle.org can connect with their account name, see attached screen shot.

===From a Mac OS X computer===
You will find some information about CVS and Mac OS X in the documentation for the complete installation package Moodle4Mac. Please read [[Complete_Install_Packages_for_Mac_OS_X#How_To_Update_Your_Moodle4Mac | How To Update Your Moodle4Mac]]. It works fine with the new CVS servers.

===Troubleshooting===

If you see something like this, make sure that there is not some firewall blocking the port (it's 2401):

$ cvs -d:pserver:anonymous@us.cvs.moodle.org:/cvsroot/moodle login
Logging in to :pserver:anonymous@us.cvs.moodle.org:2401/cvsroot/moodle
CVS password:
cvs [login aborted]: connect to us.cvs.moodle.org(130.212.64.111):2401 failed: Connection timed out

==See also==

*[[Development:Setting up Eclipse]] for step by step instructions for setting up the [http://www.eclipse.org/ Eclipse IDE] for Moodle development, which including how to do the necessary CVS operations.
* [[Development:Tracking Moodle CVS with git]]
Using Moodle forum discussions:
*[http://moodle.org/mod/forum/discuss.php?d=26731&parent=125858 Using cvs]
*[http://moodle.org/mod/forum/discuss.php?d=91891 CVS Updating of 3rd-Party Plug-ins in the Moodle folder itself]

[[Category:Administrator]]

[[fr:CVS pour administrateurs]]

Developer meeting September 2008

2008-09-19T08:26:28Z

Poltawski: /* Moodle 2.0 */

[[Developer conferences|Developer meetings]] > September 2008 meeting

*Date: 7am GMT on Friday, 19 September 2008 - see the [http://moodle.org/calendar/view.php?view=day&course=5&cal_d=19&cal_m=9&cal_y=2008 Moodle Developer Meeting calendar entry] for conversion into your time zone
*Location: [http://elluminate.remote-learner.net/join_meeting.html?meetingId=1176914355734 Elluminate Moodle Developers Meeting Room] - login using your actual name as login name and password ''moodle''

Everyone is welcome, though [http://moodle.org/mod/cvsadmin/view.php?id=7134 developers with CVS write access] will be given preference if the room becomes full.

And don't forget ..... [http://www.talklikeapirate.com/piratehome.html YARRRRRR!!!]

==Moodle 2.0==

* Demo and discussion of the new [[Development:File API|Files]] and [[Development:Repository API|Repository]] interfaces (Martin etc)
* Demo and discussion of the [[Development:Portfolio API|Portfolio]] interfaces (Penny etc)
* Demo and discussion of [[Development:Conditional activities|Activity completion]] (Sam?)
* Demo and discussion of new [[Development:Messaging 2.0|Messaging system]]
* Demo and discussion of new External Activities module, the successor to the HotPot module (Gordon)
* Demo and discussion of IMS LTI 2.0 consumer module(Ludo and J.Piguillem)...
* Wiki refactoring roadmap
* Usability testing of new features (See page comments by Gary Anderson)
* (if there's time and interest) [[https://docs.moodle.org/en/Feedback_link_for_all_elements Feedback links for most all elements]] (Jeff Forssell)
* If anyone wants to see, I can show a quick demo of current googledocs/picasa plugins portfolio/repo at the end (danp)

==Other news==

*[[GSOC/2008|Google Summer of Code 2008]] (Helen)
* OLPC School Server integrating Moodle (Martin L)
** Translation
** OUBlog stuff
** Offline moodle based on Google Gears

==See also==
*[[Roadmap]]

[[Category:Developer]]

Developer meeting September 2008

2008-09-19T08:25:56Z

Poltawski: /* Moodle 2.0 */

[[Developer conferences|Developer meetings]] > September 2008 meeting

*Date: 7am GMT on Friday, 19 September 2008 - see the [http://moodle.org/calendar/view.php?view=day&course=5&cal_d=19&cal_m=9&cal_y=2008 Moodle Developer Meeting calendar entry] for conversion into your time zone
*Location: [http://elluminate.remote-learner.net/join_meeting.html?meetingId=1176914355734 Elluminate Moodle Developers Meeting Room] - login using your actual name as login name and password ''moodle''

Everyone is welcome, though [http://moodle.org/mod/cvsadmin/view.php?id=7134 developers with CVS write access] will be given preference if the room becomes full.

And don't forget ..... [http://www.talklikeapirate.com/piratehome.html YARRRRRR!!!]

==Moodle 2.0==

* Demo and discussion of the new [[Development:File API|Files]] and [[Development:Repository API|Repository]] interfaces (Martin etc)
* Demo and discussion of the [[Development:Portfolio API|Portfolio]] interfaces (Penny etc)
* Demo and discussion of [[Development:Conditional activities|Activity completion]] (Sam?)
* Demo and discussion of new [[Development:Messaging 2.0|Messaging system]]
* Demo and discussion of new External Activities module, the successor to the HotPot module (Gordon)
* Demo and discussion of IMS LTI 2.0 consumer module(Ludo and J.Piguillem)...
* Wiki refactoring roadmap
* Usability testing of new features (See page comments by Gary Anderson)
* (if there's time and interest) [[https://docs.moodle.org/en/Feedback_link_for_all_elements Feedback links for most all elements]] (Jeff Forssell)
* If anyone wants to see, I can show a quick demo of current googledocs plugins (danp)

==Other news==

*[[GSOC/2008|Google Summer of Code 2008]] (Helen)
* OLPC School Server integrating Moodle (Martin L)
** Translation
** OUBlog stuff
** Offline moodle based on Google Gears

==See also==
*[[Roadmap]]

[[Category:Developer]]

Development talk:File API

2008-06-25T09:16:24Z

Poltawski:

Some quick questions to avoid forgetting them:

1) Will them be under the control of FileAPI (or, as they are now, fixed local storage)?

- dataroot/temp
- dataroot/lang
- dataroot/cache
- dataroot/environment
- dataroot/filter
- dataroot/rss
- dataroot/search
- dataroot/sessions
- dataroot/upgradelogs

[[User:Martin Dougiamas|Martin Dougiamas]] 03:20, 28 April 2008 (CDT) : I don't see these as being in the API - I've updated the spec.

2) Assuming we'll have a cool OOP FileAPI...

- a) Will it support different FileAPI classes (to be able to store in other systems) ?
- b) Will it support multiple FileAPI classes working together (like the Repo) ?

[[User:Martin Dougiamas|Martin Dougiamas]] 03:20, 28 April 2008 (CDT) : Hmm, I suppose it makes sense to switch the backend from local file storage to specify something else (eg database storage) but multiple File storage places doesn't make sense to me, that is the Repository API and the Portfolio API.

3) I've annotated in red some things that have sounded strange in my first look.

[[User:Martin Dougiamas|Martin Dougiamas]] 03:20, 28 April 2008 (CDT) : Thanks, all fixed.

4) Are we going to have "directory records" in the implementation, or that is going to be handled exclusively by the "moodlepath" column?

[[User:Martin Dougiamas|Martin Dougiamas]] 03:20, 28 April 2008 (CDT) : Good point. I was thinking of moodlepath only but I wonder if directory records might be more efficient. New table, I guess.

5) One general question... are we going to "force" all modules to be "autocontained" ? How are we going to handle resources, for example (with all those css, links, images..). In general how are we going to handle multiple-file packages?

[[User:Martin Dougiamas|Martin Dougiamas]] 03:20, 28 April 2008 (CDT) : they'll be a set of files, probably in a "directory" specified by moodlepath of directory record. What problems do you see? Should we retain better knowledge of the original group of files?

That's all for now, ciao4niao :-) [[User:Eloy Lafuente (stronk7)|Eloy Lafuente (stronk7)]] 21:27, 5 April 2008 (CDT)

== Making Storage 'content addressable' ==
One opportunity which this API opens up is the possibility of making the actual storage of files 'content addressable' . That is, if two users upload the same image for example, only store this file once on disk. This brings benefits in reducing the amount of storage and improving caching (especially in increasingly common situations the moodle data store served from a NFS directory or other remote storage similar). To do this we could use a hash like sha-1 on the file and store the file on disk named by its hash (rather than some arbitary id). Then when someone uploads the same file as has already been uploaded, the hash matches and we just point the database record to the same file on disk. This technique is increasingly being used by enterprise-style repositories as well as things like git. I can see the major benefits in things like scorm packages other such things which have 100's of small duplicate files stored multiple times per package and course, so sometimes you can have the same image file stored 20 different times across 20 different packages in one course, which is then duplicated for multiple classes etcetc. --[[User:Dan Poltawski|Dan Poltawski]] 07:32, 1 June 2008 (CDT)

Excellent idea, Dan, it's in [[User:Martin Dougiamas|Martin Dougiamas]] 01:43, 20 June 2008 (CDT)

== Specific File Attachments? ==

We have entries for 'moduleinstance', do we need entries to identify files per other attachments? Such as forum posts, wiki attachments, database attachments, assignment submissions?
[[User:Mike Churchward|Mike Churchward]] 16:27, 9 June 2008 (CDT)

I'm not sure if we need to have such links back to the exact forum post
or glossary entry. The idea is that the forum posts (say) would reference
the file->id. Would it be useful to have back links too ...?
[[User:Martin Dougiamas|Martin Dougiamas]] 22:30, 22 June 2008 (CDT)

==Squid==

Consider proxy support. --[[User:Helen Foster|Helen Foster]] 16:53, 9 June 2008 (CDT)

Expanding on this from the hackfest discussion - since we are hashing for storage anyway, we could
serve the sha1 hash as the Etag for every file quite easily. --[[User:Dan Poltawski|Dan Poltawski]] 04:16, 25 June 2008 (CDT)

== Batch uploads (zips) ==

Should we require that file API (optionally) require some form of batch file upload or zip/unzip function?
[[User:Mike Churchward|Mike Churchward]] 17:14, 9 June 2008 (CDT)

Yeah, it definitely needs to handle zipped files nicely ... [[User:Martin Dougiamas|Martin Dougiamas]] 22:33, 22 June 2008 (CDT)

=Skodak's rants=

The API should be split into several independent parts:
# File serving API
## file.php
## pluginfile.php
## userfile.php
## rssfile.php
# File storage API
## optional access control
## optional repo sync
# File management API
## File browsing
## File linking (editor integration)
## Upload from repository

==File serving API==
Deals with serving of files - browser requests file, Moodle sends it back. We have three main files.

===file.php===
Serves course files.
It would be nice to have some special hardcoded protection of backup files - preventing of backup file downloads/uploads; backups contain a lot of personal info, we could block restoring of backups from other sites too.

Implements basic file access. Ideally only images and files linked from course sections should be there, no XSS protection required - we expect javascript, sw, etc. there, no way to make it "secure". The access control is not critical any more if we move most most of the files into modules;

/file.php/courseid/dir/dir/filename.ext

===pluginfile.php=== (aka modfile.php)
Sends module, block, question files. Absolute file links need to be rewritten if html editing allowed in module. The links are stored internally as relative links.
* modules decide about access control
* optional XSS protection - student submitted files must not be served with normal headers, we have to force download instead; ideally there should be second wwwroot for serving of untrusted files

/pluginfile.php/contextid/arbitrary/params/or/dirs/filename.ext

pluginfile.php detects the type of plugin from context table, fetches basic info (like $course or $cm if appropriate) and calls plugin function (or later method) which does the access control and finally sends the file to user.

====blog example====
Blog entries or notes in general do not have context id (because they live in system context).
The note attachments are always served with XSS protection on, ideally we should use separate wwwroot for this. Access control can be hardcoded.

/pluginfile.php/SYSCONTEXTID/blog/blogenryid/attachmentname.ext

====assignment example====

/pluginfile.php/assignmentcontextid/submission/userid/attachmentname.ext
/pluginfile.php/assignmentcontextid/downloadall/file.zip

====scorm example====

/pluginfile.php/scormcontextid/revisionnumber/dir/somescormfile.js

The revision counter is incremented when any file changes in order to prevent caching problems. The lifetime should be adjustable in module settings.

===questions example===

pluginfile.php/SYSCONTEXTID/question/questionid/file.jpg

===userfile.php===
Personal file storage
* read/write own files only for now

/userfile.php/userid/dir/dir/filename.ext

===rssfile.php===
Replaces rss/file.php wchi is kept only for backwards compatibility.
RSS files should not require sessions/cookies, urls should contain some sort of security token/key
Internally the files may be stored in database or together with other files.
Etag support should be implemented to improve performance.

/rssfile.php/contextid/any/parameters/module/wants/rss.xml
/rssfile.php/SYSCONTEXTID/blog/userid/rss.xml

Again modules and plugins decide what gets sent to user.

==File storage API==

File contents are stored in moodledata/filepool using sha1 hashes instead of file names.

=== file table ===

This table contains one entry for every file. Enough information is kept here so that the file can be fully identified and retrieved again if necessary.

{| border="1" cellpadding="2" cellspacing="0"
|'''Field'''
|'''Type'''
|'''Default'''
|'''Info'''

|-
|'''id'''
|int(10)
|
|autoincrementing

|-
|sha1hash
|varchar(40)
|
|The sha1 hash of content.

|-
|'''contextid'''
|int(10)
|
|The context id defined in context table - identifies the instance of plugin owning the file.

|-
|instanceid
|int(10)
|
|Optional - some plugin specific instance id (eg. forum post, blog entry or assignment submission, user id for user files)

|-
|plugin
|varchar(255)
|
|The module that is the "owner" of this file (eg "moodle", "blog", "mod/assignment" or "blocks/html")

|-
|filetype
|varchar(255)
|
|Like submissions, filemanager files (images and swf linked from summaries), etc.

|-
|filename
|varchar(255)
|
|The full Unicode name of this file (case sensitive)

|-
|filepath
|text
|NULL
|Optional - relative path to file from module content root, useful in Scorm and Resource mod - most of the mods do not need this

|-
|timecreated
|int(10)
|
|The time this file was created (if known), otherwise same as time imported

|-
|timemodified
|int(10)
|
|The last time the file was modified

|-
|filesize
|int(10)
|
|size of file - bytes

|-
|userid
|int(10)
|NULL
|Optional - general id field - meaning depending on plugin
|}

index on "contextid, instanceid"

=== file_metadata table ===

This table contains extra metadata about files. Repositories could provide this, or it could be manually edited in the local copy.

{| border="1" cellpadding="2" cellspacing="0"
|'''Field'''
|'''Type'''
|'''Default'''
|'''Info'''

|-
|'''id'''
|int(10)
|
|autoincrementing

|-
|'''fileid'''
|int(10)
|
|Id of file.

|-
|'''name'''
|varchar(255)
|
|The name of extra metadata

|-
|value
|text
|
|Value

|}

=== file_access ===

This table describes optional ACL for file.

{| border="1" cellpadding="2" cellspacing="0"
|'''Field'''
|'''Type'''
|'''Default'''
|'''Info'''

|-
|'''id'''
|int(10)
|
|autoincrementing

|-
|'''fileid'''
|int(10)
|
|The file we are defining access for

|-
|'''contextid'''
|int(10)
|
|The context where this file is being published

|-
|'''capability'''
|text
|
|The capability that is required to see this file.
|}

=== repository_sync table ===

This table contains information how to synchronise data with repositories.

{| border="1" cellpadding="2" cellspacing="0"
|'''Field'''
|'''Type'''
|'''Default'''
|'''Info'''

|-
|'''id'''
|int(10)
|
|autoincrementing

|-
|'''fileid'''
|int(10)
|
|Id of file.

|-
|'''repositoryid'''
|int(10)
|
|The repository instance this is associated with, see [[Development:Repository_API]]

|-
|updates
|int(10)
|
|Specifies the update schedule (0 = none, 1 = on demand, other = some period in seconds)

|-
|repositorypath
|text
|
|The full path to the original file on the repository

|-
|timeimportfirst
|int(10)
|
|The first time this file was imported into Moodle

|-
|timeimportlast
|int(10)
|
|The most recent time that this file was imported into Moodle
|}

==File management API==

'''TODO'''

=== Some little comments to be considered (to avoid forgetting them) ===

* file_access ???? stands for ????
* each context will have its own "file manager"
* separate "file manager context" files (FMF) and "internal context" (ICF) files (current modedit files, submissions, attachements...)
* /pluginfile.php/SYSCONTEXTID/{blog|question} and so... will have own FMF too? Or only ICF ?
* rssfile.php: I'd support both Etag (cool) and Last-Modified (more used), when we receive If-None-Match/If-Modified-Since => 304
* Way to migrate
* Way to copy between contexts
* Links = -1 for them
* Deletion strategy (locks, quarantine status...)

GSOC/2008

2008-06-19T19:25:10Z

Poltawski: /* Feed aggregation library */

Overview of the [http://code.google.com/soc/2008/ Google Summer of Code 2008] projects for Moodle. Thanks to Google for the opportunity to get these developed!

==Detecting and solving specific usability issues==

The main objective of this project is to detect and solve specific usability issues.

'''Student''': [http://moodle.org/user/view.php?id=518567&course=5 Laia Subirats Maté]

'''Mentors''': [http://moodle.org/user/view.php?id=153093&course=5 David Horat] and [http://moodle.org/user/view.php?id=51473&course=5 Anthony Borrow]

'''Full specification''': [[Student projects/Usability issues|Usability issues]]

==XMLDB/SQLite==

This project is about adding SQLite to the Moodle database abstraction layer and adding a mechanism to copy a live Moodle database into a SQLite database to implement a mechanism to switch into 'testing mode' using a SQLite copy of the live database.

'''Student''': [http://moodle.org/user/view.php?id=273885&course=1 Andrei Bautu]

'''Mentor''': [http://moodle.org/user/view.php?id=17383&course=5 Penny Leach]

'''Full specification''': [[Student projects/SQLite|SQLite]]

==Animated grade statistics report==

This project is about using the [http://flare.prefuse.org/ Flare library] to develop some cool and useful visualisation of information in the Moodle gradebook.

'''Student''': [http://moodle.org/user/view.php?id=544902&course=1 Daniel Servos]

'''Mentors''': [http://moodle.org/user/view.php?id=521521&course=1 Greg Wilson], [http://moodle.org/user/view.php?id=51473&course=5 Anthony Borrow] and [http://moodle.org/user/view.php?id=240338&course=5 Nicolas Connault]

'''Full specification''': [[Student projects/Animated grade statistics report|Animated grade statistics report]]

==Web 2.0 language editing interface==

Moodle is [[Translation|designed to be translated]] into many languages, and currently has over 70 language packs. The existing [[Language editing|language editing]] interface enables translators to check for untranslated words or phrases and also enables Moodle administrators to change words or phrases used in Moodle.

This project will involve creating a Web 2.0 language editing interface which will include features such as syntax error detection, a search and replace function for changing a particular word and a spell checker.

'''Student''': [http://moodle.org/user/view.php?id=544037&course=1 Avi Mehta]

'''Mentors''': [http://moodle.org/user/view.php?id=1601&course=5 David Mudrák], [http://moodle.org/user/view.php?id=12863&course=5 Petr Skoda (Skodak)] and [http://moodle.org/user/view.php?id=3923&course=5 Koen Roggemans]

'''Full specification''': [[Student projects/Language editing interface|Language editing interface]]

==Feed aggregation library==

This project involves creating a feed aggregation library to take care of aggregating feeds (and the issues around it) and to provide them in a simple format for plugins and other core parts of Moodle to use.

'''Student''': [http://moodle.org/user/view.php?id=521708&course=1 Chris Zubak-Skees]

'''Mentor''': [http://moodle.org/user/view.php?id=104159&course=5 Dan Poltawski]

'''Full specification''': [[Student projects/Feed aggregation library|Feed aggregation library]]

'''Tracker Issue''': [http://tracker.moodle.org/browse/CONTRIB-504 CONTRIB-504]

==New customisable theme==

This project is about creating a theme for Moodle 1.9 which is customizable via a configuration page in Moodle.

'''Student''': [http://moodle.org/user/view.php?id=526259&course=1 Akshit Sharma]

'''Mentor''': [http://moodle.org/user/view.php?id=11995 Shane Elliott]

'''Full specification''': [[Student projects/Customisable theme|Customisable theme]]

==Blog improvements and the addition of a blog assignment module==

The existing [[Blog]] functionality will be improved as part of this project, and a blog assignment type developed.

'''Student''': [http://moodle.org/user/view.php?id=27192&course=5 Joey Morwick]

'''Mentor''': [http://moodle.org/user/view.php?id=423027&course=5 Mathieu Petit-Clair]

'''Full specification''': [[Student projects/Blog improvements|Blog improvements]]

==Messaging improvements==

'''Student''': [http://moodle.org/user/view.php?id=274199&course=1 Luis Filipe Romão Rodrigues]

'''Mentor''': [http://moodle.org/user/view.php?id=1 Martin Dougiamas]

'''Full specification''': [[Student projects/Further messaging improvements|Further messaging improvements]]

==Automatic accessibility checking==

'''Student''': [http://moodle.org/user/view.php?id=519485&course=1 David W. Knight]

'''Mentors''': [http://moodle.org/user/view.php?id=153093&course=5 David Horat] and [http://moodle.org/user/view.php?id=1 Martin Dougiamas]

'''Full specification''': [[Student projects/Automatic accessibility checking|Automatic accessibility checking]]

==Competency tracking==

This project is about tracking the competency of a user based on the course outcome, the desired grade and the grade that the user obtained for the course.

'''Student''': [http://moodle.org/user/view.php?id=175392&course=5 Sarves Kengatharaiyer]

'''Mentors''': [http://moodle.org/user/view.php?id=1 Martin Dougiamas] and [http://moodle.org/user/view.php?id=152257 Jonathan Newman]

'''Full specification''': [[Student projects/Competency tracking|Competency tracking]]

==Secure RSS feeds==

'''Student''': [http://moodle.org/user/view.php?id=525809&course=5 Askar Salimbaev]

'''Mentor''': [http://moodle.org/user/view.php?id=240338&course=5 Nicolas Connault]

'''Full specification''': [[Student projects/Secure RSS feeds|Secure RSS feeds]]

==Moodle IDE==

This project is about creating a Moodle integrated development environment (IDE), based on Eclipse, so that new developers can begin developing Moodle in less time.

'''Student''': [http://moodle.org/user/view.php?id=543819&course=1 Grady Laksmono]

'''Mentors''': [http://moodle.org/user/view.php?id=12863&course=5 Petr Skoda (Skodak)] and [http://moodle.org/user/view.php?id=51473&course=5 Anthony Borrow]

'''Full specification''': [[Student projects/Moodle IDE|Moodle IDE]]

[[Category:Project]]

Development talk:File API

2008-06-01T12:32:35Z

Poltawski: added thoughts on content-addressable storage

SF Developer HackFest

2008-05-30T18:14:37Z

Poltawski: Adding an idea..

[[Developer meetings]] > San Francisco 2008 Developer HackFest

*Date: 9am - 5pm Monday 9th June 2008 - see the [http://moodle.org/calendar/view.php?view=day&course=5&cal_d=9&cal_m=6&cal_y=2008 San Francisco 2008 Developer HackFest calendar entry] for conversion into your time zone
*Location: [http://www.grosvenorsfo.com/ Best Western Grosvenor Hotel], San Francisco
*Online location: [http://elluminate.remote-learner.net/join_meeting.html?meetingId=1176914355734 Elluminate Moodle Developers Meeting Room] - login using your actual name as login name and password ''moodle''

Developers and anyone wishing to become a Moodle developer can share code and ideas, and help shape the future of Moodle. This event is FREE, thanks to the sponsor [http://www.remote-learner.net/ Remote-Learner.net].

Everyone is welcome, either in person or online :-)

''Please add items to the agenda!''

==Moodle 2.0==
*[[Roadmap]] run-down (Martin)

==Gradebook Interface Improvements==
*Overview of community feedback on 1.9 gradebook (Nicolas Connault)
*Run-down of current development
*Discussion of ideas for improvement, better defaults etc.

==Other news==

*[[GSOC/2008|Google Summer of Code 2008]] (Helen)
* I (Tim Hunt, OU Moodle developer and Moodle quiz module maintainer) will be there. I don't have anything in particular that I want to say, but I am very happy to talk about anything anyone else wants to cover that I know about.

==Testing & QA==

It might be good to have a general discussion on general testing/QA issues. Eg. bug triage, encouraging more testing/QA contributors, the weekly review etc. --[[User:Dan Poltawski|Dan Poltawski]] 13:14, 30 May 2008 (CDT)

[[Category:Developer]]

Email processing

2008-04-05T16:02:20Z

Poltawski: /* Moodle configuration */

== Features ==

Moodle now makes better use of the SMTP protocol and email in general. Most of the benefits result from using a technique known as Variable Envelope Return Path (VERP).

* Works on most modern MTAs (at least on Unix systems).
* All processing of bounces and replies is secured using HMAC-MD5-8.
* Bounces are handled correctly and increase a "bad email" score for the user.
* noreply@host address is now on Reply-to field, avoiding accidental pollution of users address books.
* noreply@host has an autorresponder
* Makes it easy for modules to send emails with a signed VERP reply-to.
* Handles receving of VERP replies: Validates the HMAC-MD5-8 signature and Dispatches the encoded request data to the relevant module

== Moodle configuration ==

Edit config.php to enable bounce handling, and setup Moodle to match your MTA configuration. Here's how. Uncomment these lines in config.php (if you cannot find them, copy them from config-dist.php):

// once handlebounces is true, we will be using VERP for the return address of every sent email
$CFG->handlebounces = true;
// minimum bounces allowed per user
$CFG->minbounces = 10;
// ratio of bad emails to sent emails
// if we get more than 20% bounces
// for a given user, his/her email is marked bad
$CFG->bounceratio = .20;
// Prefix to identify your site MUST BE EXACTLY 3 characters
$CFG->mailprefix = 'mdl';
// Domain which acccepts email for processing
$CFG->maildomain = 'bounces.my.domain';

Edit the $CFG->maildomain line and one of the $CFG->mailprefix lines (the one that matches your MTA).

Make sure your server has a command-line PHP interpreter, and that it is able to connect to mysql (or postgres if relevant). If you are able to run cron.php from the commandline or from crontab, this means PHP is ok.

Edit the process_email.php script to point to the location of your php binary. It will usually be ''/usr/bin/php''.

==Email confirmation or registration words, and how to edit them.==
* Students receive a confirmation email when they create a user account. This text can be found in the lang/?/moodle.php as the emailconfirmation "variable".
* Students receive a welcome email when they enroll in a course. This text can be found in the lang/?/moodle.php file as the welcometocoursetext "variable".
*If you feel the need to edit this text, the best way to do this is to go to Site Administration -> Language editing, and then clicking on the string that you wish to edit. Once you have done this, hit the "Switch lang directory" button, and do your editing.
*If you decide to get your hands dirty and work by editing the moodle.php directly, be careful to open the file only in a simple text editor like notepad or wordpad.
*Tip: Be careful with markup - as you edit, you must use " " for things like putting in hyperlinks, titles and targets. You MUST 'escape' them by putting a \ before each ". If you don't the resulting page will appear blank. For example, the normal way to markup a hyperlink is:-
**<nowiki><a href="http://www.blah_blah">click here</a></nowiki> If you do this in the moodle.php file, your edited text won't appear.
**Use <nowiki><a href=\"http://www.blah_blah\">click here</a></nowiki>
*Use the \ before each " when marking up things like target=\"_blank\" and title=\"Blah blah\" and other similar codes.
*If you feel you must really go the route of editing the moodle.php file directly, save a copy of the file in its own folder (copy and paste the file in the same folder). That way, if it all goes wrong for you, all you have to do is delete the botched file and rename the 'copy of moodle.php' back to 'moodle.php' and you are back to square one, no harm done!
*Having said all that, it is best to do all your editing in the web interface by going to Site Administration -> Language editing, and then clicking on the string that you wish to edit.
[[Image:Example.jpg]]

== Setup under Postfix ==

Add a line to your aliases file. The line should list a 3-letter prefix, to which we'll add a '|', and the path of the script. For example for a prefix of 'mdl' and moodle installed under /var/www/moodle we have in aliases:

mdl: |/var/www/moodle/admin/process_email.php
noreply: |/var/www/moodle/admin/process_email.php

If you are using virtualdomains, consult your server administrator for the correct configuration. It probably involves editing transports and mapping your address to a "pipe" transport.

== Setup under Qmail ==

Depending on your setup, your aliases will be controlled by one or more of

* ''/etc/aliases''
* ''/var/qmail/alias/.qmail-PREFIX''

If you edit /etc/aliases add a line like this (for a prefix of 'mdl'):

mdl: |/var/www/moodle/admin/process_email.php
noreply: |/var/www/moodle/admin/process_email.php

If you create /var/qmail/alias/.qmail-PREFIX, just do

echo "|/var/www/moodle/admin/process_email.php" > /var/qmail/alias/.qmail-mdl
echo "|/var/www/moodle/admin/process_email.php" > /var/qmail/alias/.qmail-noreply

To this three letter prefix, we will add a '-' when sending and receiving messages. For more info, check out the manpage for dot-qmail.

== Setup under Exim ==

Open ''/etc/exim/exim.conf'' and add to trusted_users the user Apache and cron.php run as (usually "www-data" or "nobody").

Add a line to your aliases file. The line should list a 3-letter prefix, to which we'll add a '+', and the path of the script. For example for a prefix of 'mdl' we have in aliases:

mdl: |/var/www/moodle/admin/process_email.php
noreply: |/var/www/moodle/admin/process_email.php

If you are using virtualdomains, consult your server administrator for the correct configuration. It probably involves editing transports and mapping your address to a "pipe" transport.

More documentation about Exim can be found [http://www.exim.org/docs.html here].

You will probably have to tell Exim not to lowercase the local-part in your exim configuration. This can be done in the router which handles the mail for your aliases file with:
caseful_local_part = true

== Developer info ==

Changed functions:

* email_to_user() will set the envelope sender to a special bounce processing address (based on $CFG settings)
* email_to_user() will accept (and set) a reply-to header, to be generated by the module calling the function.
* associated string changes/additions

New functions:

* generate_email_processing_address() - ALWAYS use this to generate the reply-to header. reply-to header will look like this: (LIMIT: 64 chars total) prefix - EXACTLY four chars encodeded, packed, moduleid (0 for core) (2 chars) up to 42 chars for the modules to put anything they want it (can contain userid (or, eg for forum, postids to reply to), or anything really. 42 chars is ABSOLUTE LIMIT) 16 char hash (half an md5) of the first part of the address, together with a site "secret"

* moodle_process_email() - any non-module email processing goes here (currently used for processing bounces)

New files:

''admin/process_email.php''

This script needs to be called from your MTA for anything starting with the 3 char prefix described above (and optionally, the noreply address).

How does it work? It will break down and unencode the email address into moduleid and validate the half md5 hash, and call $modname_process_email (if it exists). Arguments to these functions are: $modargs (any part of the email address that isn't the prefix, modid or the hash) and the contents of the email (read from STDIN).

It doubles up as the noreplyaddress autorresponder if you configure it with that address as well. Replying with a friendly "this is not a real email address" message.

== Module Authors ==

Take a look at new functions moodle_process_email() and generate_email_processing_address() in moodlelib.php for ideas about how to

* encode and unencode the arguments your module needs to do the processing
* how to deal with multiple "actions" for any given module.

In a nutshell, users can send emails to Moodle using special dynamic addresses. These emails can trigger a call to a module function in the form of modulename_process_email($str, $bodyofemail). The $str part will be up to 42 characters of data generated by Moodle (presumably by your own module), and the $bodyofemailpart is the contents of the email (got by reading STDIN - usually generated by the users MUA).

The 42 characters come from the "local part" of an email address (the part before the @ sign) which can have up to 64 chars. Out of those 64 chars, Moodle uses 22 characters, leaving you with 42 characters to encode data.

What do we do with the 22 chars? Four go to the prefix, which we need to let the MTA know to pass the message to our script. Two go to identify the module ID so we know which module has generated the message (and so we dispatch the request to that module). The remaining 16 are a signature (HMAC-MD5-8) we use to authenticate the message.

Forty-two characters isn't a lot (although it could be the answer to life, the universe, and everything!) so make sure you use those characters wisely.

The most efficient way to encode database IDs in their full range (so that they can be placed in an email address) we have found is base64_encode(pack('V',2147483647)), which returns "/ / / / f w = =". The two trailing "= =" are redundant and you can remove them (you'll need to reappend them when retrieving your data). Join your parameters as encoded IDs in positional slots for efficiency.

To retrieve your data, use substr() to separate your parameters, and then unpack('V',base64_decode($str)). Note that it'll return a one-element array.
Note:

Using 'V' reaches 2147483647, half the range of mySQL's INT. Additionally, 'V' behaves like a signed value, rather than an unsigned, so I suspect there's a bug in PHP's documentation of pack().

With each ID taking 6-chars (8 chars if we find a way to use the full range of 'V'), you have a limited number of parameters. If you need to encode more information, store it in the DB and send emails that point to your stored data. Remember to cleanup this temporary data after a safe period of time.

Note:

Do not try to use variable-width encoding to put IDs, as it'll work in small installations and break in larger ones.

== Security issues ==

Any code in modulename_process_email() _must_ assume it will see repeat replies and handle them gracefully. The definition of 'gracefully' depends on what the code does.

Email servers (MTAs) will sometimes re-transmit a message if they are unsure that the receiving MTA got it -- and sysadmins may sometimes replay a whole email queue if something's gone wrong. In that case, they email body will be identical, the headers slightly different.

In a different case, the user may 'reply' to the message twice. Perhaps in error, perhaps purposefully. What to do depends on the specific scenario.

We /could/ support better protection at the framework level, by keeping track of every reply-to address we send out. We decided against that because (a) the performance impact will be important (b) we want the 1st cut to be lightweight and simple to change in case we need to.

With this the initial implementation, modules should expose functions that handle these "replay" cases correctly. If later we want to expose additional functions, we can add such tracking as an optional thing. It'd be awful to have it across all emails sent from Moodle.

== Experimental email processing, pending merge into 1.9/2.0 ==
===Module Authors===
The email interface allows you to let users respond to emails sent via your module and affect change in them. The basic idea is that each email is uniquely identified with an email session that carries with it a simple payload. This payload is stored via database, and cannot be changed by the user. Replies are handled by module code in your lib.php library - modulename_process_email($payload, $body) - which is passed the session payload and the body (with full headers) of the email received, to be parsed as you like. Some utility functions are available as well to help with this.

Here is an outline of the intended use:
* Before sending an email to a user via the email_to_user() function, the module should check to see if the config variable $CFG->emailinterface is enabled. (This is off by default)
* Assuming the email interface is enabled, the function generate_email_verp_address($moduleid, $payload, $userid) should be called to create an "email session". The payload should include any data you wish to use in your handling routine - such as the id of the user, or what part of the interface he is interacting with. The function will return a string containing the email session id.
* Call the email_to_user() function as normal, appending the email session id as the 11th parameter. Keep in mind that you will probably also need to change the text or html of the email to point out that the email interface is active.
* A function contained in the lib.php file of your module, named [modulename]_process_email($payload, $body) should be written. This is the handler function, which is called when an email reply is received. This function can inspect the contents of the payload received, then parse the email body for data. The $body parameter returns the full email, including the headers.

Note that each email can only be replied to once - the email session is destroyed once an email has been received. If you wish to have multiple replies from a user, you will have to create a new session (and new email) every time. The email session table is also regularly pruned, the default settings allow sessions to persist for one month.

The following utility functions are also defined in admin/process_email.php, which can help:
* get_email_subject($fullemail) - return the subject of an email
* strip_email_headers($fullemail) - strip all headers off an email
* email_is_multipart($fullemail) - determine if an email is a multipart message
* seperate_multipart($multipartmsg) - seperate a multipart email message into an array of parts, each of which has headers and a body.
* get_multipart_content_type($part) - determine the content-type of a multipart, generally text/plain or text/html.
* multipart_is_quoted_printable($part) - determine if a email part has been encoded with the quoted-printable encoding - some email clients tend to do this to html.
* decode_quoted_printable($partdata) - decode a email that has been encoded with quoted-printable encoding - this needs to be run on the data section of a part (strip_email_headers can be used to seperate the headers of a part).
* strip_email_reply($body, $ishtml, $identifier) - strip all irrelevant lines of an email message based on a unique identifier embedded within the original message. This is used to determine what is user input, and what is irrelevant data the email client has added.
* strip_email_reply_multi($body, $ishtml, $identifier) - as above, but allowing for multiple sections of possible user input in a single email message.

Examples of use of these functions can be seen in the handler for the forum module. The idea behind the strip reply functions is that the original email will contain a unique identifier, such as a random 10 character string. This is embedded in a line with instructions for the user to write below and above it, e.g:
\/ Please put your response here [ABCDEFGHIJ] \/
<User Data>
/\ Please put your response here [ABCDEFGHIJ] /\
This is necessary because of the wide differences between webmail and email clients in handling quotes and prefixing dates/times to replied emails. Keep in mind that text emails have an 80-character line width restrictions, which must also include the quote added by the email client, so any line of a text-email with this identifier should be kept to at least 77 characters to avoid wrapping by an email client. This unique identifier can be easily stored in the payload. Multiple input sections are handled similarly - a number is appended to the end of the identifier, e.g. [ABCDEFGHIJ1] to signfiy the first section of input. The strip_email_reply_multi returns an array containing a map from each numbered section it finds to a string containing all data in that section.

===Developer Info===
The reply-to header is now encoded in base32 instead of base64 to allow for case-insensitive MTA's. Generate_email_processing_address was left as a backwards-compatibility wrapper, modules should use generate_email_verp_address now instead.

Email sessions have been created to deal with the lack of space avaliable in the header under base32, payloads are stored in a new table mdl_email_sessions, and are identified with a unique base32 key which is embedded in the reply-to header. Each row has a timestamp, and the table is pruned during cleanup cron in admin/cron.php.

Bounce detection is now linked in with the email interface - bounce detection is included in admin/process_email.php. Some new configuration variables have also been created to help configure these changes - $CFG->emailinterface and $CFG->emailsessiontime - and both have been added to the server settings admin page.

[[Category:Administrator]]
[[Category:Developer]]

Email processing

2008-04-05T15:20:03Z

Poltawski: added exim configuration option for keeping case in local part

== Features ==

Moodle now makes better use of the SMTP protocol and email in general. Most of the benefits result from using a technique known as Variable Envelope Return Path (VERP).

* Works on most modern MTAs (at least on Unix systems).
* All processing of bounces and replies is secured using HMAC-MD5-8.
* Bounces are handled correctly and increase a "bad email" score for the user.
* noreply@host address is now on Reply-to field, avoiding accidental pollution of users address books.
* noreply@host has an autorresponder
* Makes it easy for modules to send emails with a signed VERP reply-to.
* Handles receving of VERP replies: Validates the HMAC-MD5-8 signature and Dispatches the encoded request data to the relevant module

== Moodle configuration ==

Edit config.php to enable bounce handling, and setup Moodle to match your MTA configuration. Here's how. Uncomment these lines in config.php (if you cannot find them, copy them from config-dist.php):

// once handlebounces is true, we will be using VERP for the return address of every sent email
$CFG->handlebounces = true;
// minimum bounces allowed per user
$CFG->minbounces = 10;
// ratio of bad emails to sent emails
// if we get more than 20% bounces
// for a given user, his/her email is marked bad
$CFG->bounceratio = .20;
// Prefix to identify your site MUST BE EXACTLY 4 characters
$CFG->mailprefix = 'mdl1';
// Domain which acccepts email for processing
$CFG->maildomain = 'bounces.my.domain';

Edit the $CFG->maildomain line and one of the $CFG->mailprefix lines (the one that matches your MTA).

Make sure your server has a command-line PHP interpreter, and that it is able to connect to mysql (or postgres if relevant). If you are able to run cron.php from the commandline or from crontab, this means PHP is ok.

Edit the process_email.php script to point to the location of your php binary. It will usually be ''/usr/bin/php''.

==Email confirmation or registration words, and how to edit them.==
* Students receive a confirmation email when they create a user account. This text can be found in the lang/?/moodle.php as the emailconfirmation "variable".
* Students receive a welcome email when they enroll in a course. This text can be found in the lang/?/moodle.php file as the welcometocoursetext "variable".
*If you feel the need to edit this text, the best way to do this is to go to Site Administration -> Language editing, and then clicking on the string that you wish to edit. Once you have done this, hit the "Switch lang directory" button, and do your editing.
*If you decide to get your hands dirty and work by editing the moodle.php directly, be careful to open the file only in a simple text editor like notepad or wordpad.
*Tip: Be careful with markup - as you edit, you must use " " for things like putting in hyperlinks, titles and targets. You MUST 'escape' them by putting a \ before each ". If you don't the resulting page will appear blank. For example, the normal way to markup a hyperlink is:-
**<nowiki><a href="http://www.blah_blah">click here</a></nowiki> If you do this in the moodle.php file, your edited text won't appear.
**Use <nowiki><a href=\"http://www.blah_blah\">click here</a></nowiki>
*Use the \ before each " when marking up things like target=\"_blank\" and title=\"Blah blah\" and other similar codes.
*If you feel you must really go the route of editing the moodle.php file directly, save a copy of the file in its own folder (copy and paste the file in the same folder). That way, if it all goes wrong for you, all you have to do is delete the botched file and rename the 'copy of moodle.php' back to 'moodle.php' and you are back to square one, no harm done!
*Having said all that, it is best to do all your editing in the web interface by going to Site Administration -> Language editing, and then clicking on the string that you wish to edit.
[[Image:Example.jpg]]

== Setup under Postfix ==

Add a line to your aliases file. The line should list a 3-letter prefix, to which we'll add a '|', and the path of the script. For example for a prefix of 'mdl' and moodle installed under /var/www/moodle we have in aliases:

mdl: |/var/www/moodle/admin/process_email.php
noreply: |/var/www/moodle/admin/process_email.php

If you are using virtualdomains, consult your server administrator for the correct configuration. It probably involves editing transports and mapping your address to a "pipe" transport.

== Setup under Qmail ==

Depending on your setup, your aliases will be controlled by one or more of

* ''/etc/aliases''
* ''/var/qmail/alias/.qmail-PREFIX''

If you edit /etc/aliases add a line like this (for a prefix of 'mdl'):

mdl: |/var/www/moodle/admin/process_email.php
noreply: |/var/www/moodle/admin/process_email.php

If you create /var/qmail/alias/.qmail-PREFIX, just do

echo "|/var/www/moodle/admin/process_email.php" > /var/qmail/alias/.qmail-mdl
echo "|/var/www/moodle/admin/process_email.php" > /var/qmail/alias/.qmail-noreply

To this three letter prefix, we will add a '-' when sending and receiving messages. For more info, check out the manpage for dot-qmail.

== Setup under Exim ==

Open ''/etc/exim/exim.conf'' and add to trusted_users the user Apache and cron.php run as (usually "www-data" or "nobody").

Add a line to your aliases file. The line should list a 3-letter prefix, to which we'll add a '+', and the path of the script. For example for a prefix of 'mdl' we have in aliases:

mdl: |/var/www/moodle/admin/process_email.php
noreply: |/var/www/moodle/admin/process_email.php

If you are using virtualdomains, consult your server administrator for the correct configuration. It probably involves editing transports and mapping your address to a "pipe" transport.

More documentation about Exim can be found [http://www.exim.org/docs.html here].

You will probably have to tell Exim not to lowercase the local-part in your exim configuration. This can be done in the router which handles the mail for your aliases file with:
caseful_local_part = true

== Developer info ==

Changed functions:

* email_to_user() will set the envelope sender to a special bounce processing address (based on $CFG settings)
* email_to_user() will accept (and set) a reply-to header, to be generated by the module calling the function.
* associated string changes/additions

New functions:

* generate_email_processing_address() - ALWAYS use this to generate the reply-to header. reply-to header will look like this: (LIMIT: 64 chars total) prefix - EXACTLY four chars encodeded, packed, moduleid (0 for core) (2 chars) up to 42 chars for the modules to put anything they want it (can contain userid (or, eg for forum, postids to reply to), or anything really. 42 chars is ABSOLUTE LIMIT) 16 char hash (half an md5) of the first part of the address, together with a site "secret"

* moodle_process_email() - any non-module email processing goes here (currently used for processing bounces)

New files:

''admin/process_email.php''

This script needs to be called from your MTA for anything starting with the 3 char prefix described above (and optionally, the noreply address).

How does it work? It will break down and unencode the email address into moduleid and validate the half md5 hash, and call $modname_process_email (if it exists). Arguments to these functions are: $modargs (any part of the email address that isn't the prefix, modid or the hash) and the contents of the email (read from STDIN).

It doubles up as the noreplyaddress autorresponder if you configure it with that address as well. Replying with a friendly "this is not a real email address" message.

== Module Authors ==

Take a look at new functions moodle_process_email() and generate_email_processing_address() in moodlelib.php for ideas about how to

* encode and unencode the arguments your module needs to do the processing
* how to deal with multiple "actions" for any given module.

In a nutshell, users can send emails to Moodle using special dynamic addresses. These emails can trigger a call to a module function in the form of modulename_process_email($str, $bodyofemail). The $str part will be up to 42 characters of data generated by Moodle (presumably by your own module), and the $bodyofemailpart is the contents of the email (got by reading STDIN - usually generated by the users MUA).

The 42 characters come from the "local part" of an email address (the part before the @ sign) which can have up to 64 chars. Out of those 64 chars, Moodle uses 22 characters, leaving you with 42 characters to encode data.

What do we do with the 22 chars? Four go to the prefix, which we need to let the MTA know to pass the message to our script. Two go to identify the module ID so we know which module has generated the message (and so we dispatch the request to that module). The remaining 16 are a signature (HMAC-MD5-8) we use to authenticate the message.

Forty-two characters isn't a lot (although it could be the answer to life, the universe, and everything!) so make sure you use those characters wisely.

The most efficient way to encode database IDs in their full range (so that they can be placed in an email address) we have found is base64_encode(pack('V',2147483647)), which returns "/ / / / f w = =". The two trailing "= =" are redundant and you can remove them (you'll need to reappend them when retrieving your data). Join your parameters as encoded IDs in positional slots for efficiency.

To retrieve your data, use substr() to separate your parameters, and then unpack('V',base64_decode($str)). Note that it'll return a one-element array.
Note:

Using 'V' reaches 2147483647, half the range of mySQL's INT. Additionally, 'V' behaves like a signed value, rather than an unsigned, so I suspect there's a bug in PHP's documentation of pack().

With each ID taking 6-chars (8 chars if we find a way to use the full range of 'V'), you have a limited number of parameters. If you need to encode more information, store it in the DB and send emails that point to your stored data. Remember to cleanup this temporary data after a safe period of time.

Note:

Do not try to use variable-width encoding to put IDs, as it'll work in small installations and break in larger ones.

== Security issues ==

Any code in modulename_process_email() _must_ assume it will see repeat replies and handle them gracefully. The definition of 'gracefully' depends on what the code does.

Email servers (MTAs) will sometimes re-transmit a message if they are unsure that the receiving MTA got it -- and sysadmins may sometimes replay a whole email queue if something's gone wrong. In that case, they email body will be identical, the headers slightly different.

In a different case, the user may 'reply' to the message twice. Perhaps in error, perhaps purposefully. What to do depends on the specific scenario.

We /could/ support better protection at the framework level, by keeping track of every reply-to address we send out. We decided against that because (a) the performance impact will be important (b) we want the 1st cut to be lightweight and simple to change in case we need to.

With this the initial implementation, modules should expose functions that handle these "replay" cases correctly. If later we want to expose additional functions, we can add such tracking as an optional thing. It'd be awful to have it across all emails sent from Moodle.

== Experimental email processing, pending merge into 1.9/2.0 ==
===Module Authors===
The email interface allows you to let users respond to emails sent via your module and affect change in them. The basic idea is that each email is uniquely identified with an email session that carries with it a simple payload. This payload is stored via database, and cannot be changed by the user. Replies are handled by module code in your lib.php library - modulename_process_email($payload, $body) - which is passed the session payload and the body (with full headers) of the email received, to be parsed as you like. Some utility functions are available as well to help with this.

Here is an outline of the intended use:
* Before sending an email to a user via the email_to_user() function, the module should check to see if the config variable $CFG->emailinterface is enabled. (This is off by default)
* Assuming the email interface is enabled, the function generate_email_verp_address($moduleid, $payload, $userid) should be called to create an "email session". The payload should include any data you wish to use in your handling routine - such as the id of the user, or what part of the interface he is interacting with. The function will return a string containing the email session id.
* Call the email_to_user() function as normal, appending the email session id as the 11th parameter. Keep in mind that you will probably also need to change the text or html of the email to point out that the email interface is active.
* A function contained in the lib.php file of your module, named [modulename]_process_email($payload, $body) should be written. This is the handler function, which is called when an email reply is received. This function can inspect the contents of the payload received, then parse the email body for data. The $body parameter returns the full email, including the headers.

Note that each email can only be replied to once - the email session is destroyed once an email has been received. If you wish to have multiple replies from a user, you will have to create a new session (and new email) every time. The email session table is also regularly pruned, the default settings allow sessions to persist for one month.

The following utility functions are also defined in admin/process_email.php, which can help:
* get_email_subject($fullemail) - return the subject of an email
* strip_email_headers($fullemail) - strip all headers off an email
* email_is_multipart($fullemail) - determine if an email is a multipart message
* seperate_multipart($multipartmsg) - seperate a multipart email message into an array of parts, each of which has headers and a body.
* get_multipart_content_type($part) - determine the content-type of a multipart, generally text/plain or text/html.
* multipart_is_quoted_printable($part) - determine if a email part has been encoded with the quoted-printable encoding - some email clients tend to do this to html.
* decode_quoted_printable($partdata) - decode a email that has been encoded with quoted-printable encoding - this needs to be run on the data section of a part (strip_email_headers can be used to seperate the headers of a part).
* strip_email_reply($body, $ishtml, $identifier) - strip all irrelevant lines of an email message based on a unique identifier embedded within the original message. This is used to determine what is user input, and what is irrelevant data the email client has added.
* strip_email_reply_multi($body, $ishtml, $identifier) - as above, but allowing for multiple sections of possible user input in a single email message.

Examples of use of these functions can be seen in the handler for the forum module. The idea behind the strip reply functions is that the original email will contain a unique identifier, such as a random 10 character string. This is embedded in a line with instructions for the user to write below and above it, e.g:
\/ Please put your response here [ABCDEFGHIJ] \/
<User Data>
/\ Please put your response here [ABCDEFGHIJ] /\
This is necessary because of the wide differences between webmail and email clients in handling quotes and prefixing dates/times to replied emails. Keep in mind that text emails have an 80-character line width restrictions, which must also include the quote added by the email client, so any line of a text-email with this identifier should be kept to at least 77 characters to avoid wrapping by an email client. This unique identifier can be easily stored in the payload. Multiple input sections are handled similarly - a number is appended to the end of the identifier, e.g. [ABCDEFGHIJ1] to signfiy the first section of input. The strip_email_reply_multi returns an array containing a map from each numbered section it finds to a string containing all data in that section.

===Developer Info===
The reply-to header is now encoded in base32 instead of base64 to allow for case-insensitive MTA's. Generate_email_processing_address was left as a backwards-compatibility wrapper, modules should use generate_email_verp_address now instead.

Email sessions have been created to deal with the lack of space avaliable in the header under base32, payloads are stored in a new table mdl_email_sessions, and are identified with a unique base32 key which is embedded in the reply-to header. Each row has a timestamp, and the table is pruned during cleanup cron in admin/cron.php.

Bounce detection is now linked in with the email interface - bounce detection is included in admin/process_email.php. Some new configuration variables have also been created to help configure these changes - $CFG->emailinterface and $CFG->emailsessiontime - and both have been added to the server settings admin page.

[[Category:Administrator]]
[[Category:Developer]]

Email processing

2008-04-05T15:04:40Z

Poltawski: put example config options for malprefix/domain

== Features ==

Moodle now makes better use of the SMTP protocol and email in general. Most of the benefits result from using a technique known as Variable Envelope Return Path (VERP).

* Works on most modern MTAs (at least on Unix systems).
* All processing of bounces and replies is secured using HMAC-MD5-8.
* Bounces are handled correctly and increase a "bad email" score for the user.
* noreply@host address is now on Reply-to field, avoiding accidental pollution of users address books.
* noreply@host has an autorresponder
* Makes it easy for modules to send emails with a signed VERP reply-to.
* Handles receving of VERP replies: Validates the HMAC-MD5-8 signature and Dispatches the encoded request data to the relevant module

== Moodle configuration ==

Edit config.php to enable bounce handling, and setup Moodle to match your MTA configuration. Here's how. Uncomment these lines in config.php (if you cannot find them, copy them from config-dist.php):

// once handlebounces is true, we will be using VERP for the return address of every sent email
$CFG->handlebounces = true;
// minimum bounces allowed per user
$CFG->minbounces = 10;
// ratio of bad emails to sent emails
// if we get more than 20% bounces
// for a given user, his/her email is marked bad
$CFG->bounceratio = .20;
// Prefix to identify your site MUST BE EXACTLY 4 characters
$CFG->mailprefix = 'mdl1';
// Domain which acccepts email for processing
$CFG->maildomain = 'bounces.my.domain';

Edit the $CFG->maildomain line and one of the $CFG->mailprefix lines (the one that matches your MTA).

Make sure your server has a command-line PHP interpreter, and that it is able to connect to mysql (or postgres if relevant). If you are able to run cron.php from the commandline or from crontab, this means PHP is ok.

Edit the process_email.php script to point to the location of your php binary. It will usually be ''/usr/bin/php''.

==Email confirmation or registration words, and how to edit them.==
* Students receive a confirmation email when they create a user account. This text can be found in the lang/?/moodle.php as the emailconfirmation "variable".
* Students receive a welcome email when they enroll in a course. This text can be found in the lang/?/moodle.php file as the welcometocoursetext "variable".
*If you feel the need to edit this text, the best way to do this is to go to Site Administration -> Language editing, and then clicking on the string that you wish to edit. Once you have done this, hit the "Switch lang directory" button, and do your editing.
*If you decide to get your hands dirty and work by editing the moodle.php directly, be careful to open the file only in a simple text editor like notepad or wordpad.
*Tip: Be careful with markup - as you edit, you must use " " for things like putting in hyperlinks, titles and targets. You MUST 'escape' them by putting a \ before each ". If you don't the resulting page will appear blank. For example, the normal way to markup a hyperlink is:-
**<nowiki><a href="http://www.blah_blah">click here</a></nowiki> If you do this in the moodle.php file, your edited text won't appear.
**Use <nowiki><a href=\"http://www.blah_blah\">click here</a></nowiki>
*Use the \ before each " when marking up things like target=\"_blank\" and title=\"Blah blah\" and other similar codes.
*If you feel you must really go the route of editing the moodle.php file directly, save a copy of the file in its own folder (copy and paste the file in the same folder). That way, if it all goes wrong for you, all you have to do is delete the botched file and rename the 'copy of moodle.php' back to 'moodle.php' and you are back to square one, no harm done!
*Having said all that, it is best to do all your editing in the web interface by going to Site Administration -> Language editing, and then clicking on the string that you wish to edit.
[[Image:Example.jpg]]

== Setup under Postfix ==

Add a line to your aliases file. The line should list a 3-letter prefix, to which we'll add a '|', and the path of the script. For example for a prefix of 'mdl' and moodle installed under /var/www/moodle we have in aliases:

mdl: |/var/www/moodle/admin/process_email.php
noreply: |/var/www/moodle/admin/process_email.php

If you are using virtualdomains, consult your server administrator for the correct configuration. It probably involves editing transports and mapping your address to a "pipe" transport.

== Setup under Qmail ==

Depending on your setup, your aliases will be controlled by one or more of

* ''/etc/aliases''
* ''/var/qmail/alias/.qmail-PREFIX''

If you edit /etc/aliases add a line like this (for a prefix of 'mdl'):

mdl: |/var/www/moodle/admin/process_email.php
noreply: |/var/www/moodle/admin/process_email.php

If you create /var/qmail/alias/.qmail-PREFIX, just do

echo "|/var/www/moodle/admin/process_email.php" > /var/qmail/alias/.qmail-mdl
echo "|/var/www/moodle/admin/process_email.php" > /var/qmail/alias/.qmail-noreply

To this three letter prefix, we will add a '-' when sending and receiving messages. For more info, check out the manpage for dot-qmail.

== Setup under Exim ==

Open ''/etc/exim/exim.conf'' and add to trusted_users the user Apache and cron.php run as (usually "www-data" or "nobody").

Add a line to your aliases file. The line should list a 3-letter prefix, to which we'll add a '+', and the path of the script. For example for a prefix of 'mdl' we have in aliases:

mdl: |/var/www/moodle/admin/process_email.php
noreply: |/var/www/moodle/admin/process_email.php

If you are using virtualdomains, consult your server administrator for the correct configuration. It probably involves editing transports and mapping your address to a "pipe" transport.

More documentation about Exim can be found [http://www.exim.org/docs.html here]. You may have to tell Exim not to lowercase the local-part.

== Developer info ==

Changed functions:

* email_to_user() will set the envelope sender to a special bounce processing address (based on $CFG settings)
* email_to_user() will accept (and set) a reply-to header, to be generated by the module calling the function.
* associated string changes/additions

New functions:

* generate_email_processing_address() - ALWAYS use this to generate the reply-to header. reply-to header will look like this: (LIMIT: 64 chars total) prefix - EXACTLY four chars encodeded, packed, moduleid (0 for core) (2 chars) up to 42 chars for the modules to put anything they want it (can contain userid (or, eg for forum, postids to reply to), or anything really. 42 chars is ABSOLUTE LIMIT) 16 char hash (half an md5) of the first part of the address, together with a site "secret"

* moodle_process_email() - any non-module email processing goes here (currently used for processing bounces)

New files:

''admin/process_email.php''

This script needs to be called from your MTA for anything starting with the 3 char prefix described above (and optionally, the noreply address).

How does it work? It will break down and unencode the email address into moduleid and validate the half md5 hash, and call $modname_process_email (if it exists). Arguments to these functions are: $modargs (any part of the email address that isn't the prefix, modid or the hash) and the contents of the email (read from STDIN).

It doubles up as the noreplyaddress autorresponder if you configure it with that address as well. Replying with a friendly "this is not a real email address" message.

== Module Authors ==

Take a look at new functions moodle_process_email() and generate_email_processing_address() in moodlelib.php for ideas about how to

* encode and unencode the arguments your module needs to do the processing
* how to deal with multiple "actions" for any given module.

In a nutshell, users can send emails to Moodle using special dynamic addresses. These emails can trigger a call to a module function in the form of modulename_process_email($str, $bodyofemail). The $str part will be up to 42 characters of data generated by Moodle (presumably by your own module), and the $bodyofemailpart is the contents of the email (got by reading STDIN - usually generated by the users MUA).

The 42 characters come from the "local part" of an email address (the part before the @ sign) which can have up to 64 chars. Out of those 64 chars, Moodle uses 22 characters, leaving you with 42 characters to encode data.

What do we do with the 22 chars? Four go to the prefix, which we need to let the MTA know to pass the message to our script. Two go to identify the module ID so we know which module has generated the message (and so we dispatch the request to that module). The remaining 16 are a signature (HMAC-MD5-8) we use to authenticate the message.

Forty-two characters isn't a lot (although it could be the answer to life, the universe, and everything!) so make sure you use those characters wisely.

The most efficient way to encode database IDs in their full range (so that they can be placed in an email address) we have found is base64_encode(pack('V',2147483647)), which returns "/ / / / f w = =". The two trailing "= =" are redundant and you can remove them (you'll need to reappend them when retrieving your data). Join your parameters as encoded IDs in positional slots for efficiency.

To retrieve your data, use substr() to separate your parameters, and then unpack('V',base64_decode($str)). Note that it'll return a one-element array.
Note:

Using 'V' reaches 2147483647, half the range of mySQL's INT. Additionally, 'V' behaves like a signed value, rather than an unsigned, so I suspect there's a bug in PHP's documentation of pack().

With each ID taking 6-chars (8 chars if we find a way to use the full range of 'V'), you have a limited number of parameters. If you need to encode more information, store it in the DB and send emails that point to your stored data. Remember to cleanup this temporary data after a safe period of time.

Note:

Do not try to use variable-width encoding to put IDs, as it'll work in small installations and break in larger ones.

== Security issues ==

Any code in modulename_process_email() _must_ assume it will see repeat replies and handle them gracefully. The definition of 'gracefully' depends on what the code does.

Email servers (MTAs) will sometimes re-transmit a message if they are unsure that the receiving MTA got it -- and sysadmins may sometimes replay a whole email queue if something's gone wrong. In that case, they email body will be identical, the headers slightly different.

In a different case, the user may 'reply' to the message twice. Perhaps in error, perhaps purposefully. What to do depends on the specific scenario.

We /could/ support better protection at the framework level, by keeping track of every reply-to address we send out. We decided against that because (a) the performance impact will be important (b) we want the 1st cut to be lightweight and simple to change in case we need to.

With this the initial implementation, modules should expose functions that handle these "replay" cases correctly. If later we want to expose additional functions, we can add such tracking as an optional thing. It'd be awful to have it across all emails sent from Moodle.

== Experimental email processing, pending merge into 1.9/2.0 ==
===Module Authors===
The email interface allows you to let users respond to emails sent via your module and affect change in them. The basic idea is that each email is uniquely identified with an email session that carries with it a simple payload. This payload is stored via database, and cannot be changed by the user. Replies are handled by module code in your lib.php library - modulename_process_email($payload, $body) - which is passed the session payload and the body (with full headers) of the email received, to be parsed as you like. Some utility functions are available as well to help with this.

Here is an outline of the intended use:
* Before sending an email to a user via the email_to_user() function, the module should check to see if the config variable $CFG->emailinterface is enabled. (This is off by default)
* Assuming the email interface is enabled, the function generate_email_verp_address($moduleid, $payload, $userid) should be called to create an "email session". The payload should include any data you wish to use in your handling routine - such as the id of the user, or what part of the interface he is interacting with. The function will return a string containing the email session id.
* Call the email_to_user() function as normal, appending the email session id as the 11th parameter. Keep in mind that you will probably also need to change the text or html of the email to point out that the email interface is active.
* A function contained in the lib.php file of your module, named [modulename]_process_email($payload, $body) should be written. This is the handler function, which is called when an email reply is received. This function can inspect the contents of the payload received, then parse the email body for data. The $body parameter returns the full email, including the headers.

Note that each email can only be replied to once - the email session is destroyed once an email has been received. If you wish to have multiple replies from a user, you will have to create a new session (and new email) every time. The email session table is also regularly pruned, the default settings allow sessions to persist for one month.

The following utility functions are also defined in admin/process_email.php, which can help:
* get_email_subject($fullemail) - return the subject of an email
* strip_email_headers($fullemail) - strip all headers off an email
* email_is_multipart($fullemail) - determine if an email is a multipart message
* seperate_multipart($multipartmsg) - seperate a multipart email message into an array of parts, each of which has headers and a body.
* get_multipart_content_type($part) - determine the content-type of a multipart, generally text/plain or text/html.
* multipart_is_quoted_printable($part) - determine if a email part has been encoded with the quoted-printable encoding - some email clients tend to do this to html.
* decode_quoted_printable($partdata) - decode a email that has been encoded with quoted-printable encoding - this needs to be run on the data section of a part (strip_email_headers can be used to seperate the headers of a part).
* strip_email_reply($body, $ishtml, $identifier) - strip all irrelevant lines of an email message based on a unique identifier embedded within the original message. This is used to determine what is user input, and what is irrelevant data the email client has added.
* strip_email_reply_multi($body, $ishtml, $identifier) - as above, but allowing for multiple sections of possible user input in a single email message.

Examples of use of these functions can be seen in the handler for the forum module. The idea behind the strip reply functions is that the original email will contain a unique identifier, such as a random 10 character string. This is embedded in a line with instructions for the user to write below and above it, e.g:
\/ Please put your response here [ABCDEFGHIJ] \/
<User Data>
/\ Please put your response here [ABCDEFGHIJ] /\
This is necessary because of the wide differences between webmail and email clients in handling quotes and prefixing dates/times to replied emails. Keep in mind that text emails have an 80-character line width restrictions, which must also include the quote added by the email client, so any line of a text-email with this identifier should be kept to at least 77 characters to avoid wrapping by an email client. This unique identifier can be easily stored in the payload. Multiple input sections are handled similarly - a number is appended to the end of the identifier, e.g. [ABCDEFGHIJ1] to signfiy the first section of input. The strip_email_reply_multi returns an array containing a map from each numbered section it finds to a string containing all data in that section.

===Developer Info===
The reply-to header is now encoded in base32 instead of base64 to allow for case-insensitive MTA's. Generate_email_processing_address was left as a backwards-compatibility wrapper, modules should use generate_email_verp_address now instead.

Email sessions have been created to deal with the lack of space avaliable in the header under base32, payloads are stored in a new table mdl_email_sessions, and are identified with a unique base32 key which is embedded in the reply-to header. Each row has a timestamp, and the table is pruned during cleanup cron in admin/cron.php.

Bounce detection is now linked in with the email interface - bounce detection is included in admin/process_email.php. Some new configuration variables have also been created to help configure these changes - $CFG->emailinterface and $CFG->emailsessiontime - and both have been added to the server settings admin page.

[[Category:Administrator]]
[[Category:Developer]]

Projects for new developers

2008-03-24T23:02:43Z

Poltawski: adding sqlite project, for the adventurous

This page lists possible student projects in Moodle, in particular suggestions for GSOC 2008 projects.
__NOTOC__
===Potential GSOC students===

Firstly, review the [http://code.google.com/p/google-summer-of-code/wiki/AdviceforStudents GSOC advice for students].

You may choose a project from the list below and/or suggest your own project ideas. Please use the [http://moodle.org/mod/forum/view.php?id=7105 Moodle.org Student projects forum] for discussing project ideas.

Student applications start on Monday 24th March 2008. See the [http://code.google.com/opensource/gsoc/2008/faqs.html#0.1_timeline GSOC timeline] for more information.

===Potential mentors===

Moodle developers willing to mentor a GSOC project, please suggest a new project under the heading "Ideas we're considering for GSOC" and/or add your name to an existing project. Contact [http://moodle.org/user/view.php?id=24152&course=5 Helen Foster] if you have any questions.

__TOC__
==Useful links==

* [[GSOC]] - describing Moodle's involvement with Google in their Summer of Code program, including a list of GSOC projects from 2006 and 2007
* [http://tracker.moodle.org/secure/IssueNavigator.jspa?mode=hide&requestId=10512 Popular Requests] - Most voted-for enhancements and new feature requests in the tracker

==GSOC suggested projects==

Following a review, approved suggested projects which have mentors will be moved under this heading. ''More approved suggested projects coming soon!''

=== Consuming RSS feeds ===

RSS/Atom feeds are becoming an important technology on the web and so it's crucial that Moodle has good support for consuming these feeds for a variety of different uses. At present we generate [[RSS|RSS feeds]] for use by other applications, though only consume feeds in the [[RSS feeds block]]. It would be useful to have a core library which can take care of aggregating feeds (and the issues around it) and to provide them in a simple format for plugins and other core parts of Moodle to use.

This project will involve creating a feed aggregation library which:
* Will consume most current aggregation technologies (RSS 0.90/1.0/2.0 /Atom etc)
* Will work behind a variety of web proxies
* Periodically aggregates and uses a caching mechanism (doesn't query remote server with every request)
* (Is probably based on an existing PHP framework)

In order test this library, it will be necessary to refactor the RSS block to use it.

'''Mentor:''' [http://moodle.org/user/view.php?id=104159&course=5 Dan Poltawski]

=== Secure RSS feeds ===

Currently RSS is less than useful because:
* Either we can't publish private information to the outside world because it's too sensitive.
* We open up sensitive information to the outside world

We should add codes to make the RSS URLs practically impossible to guess, much the same way as Google Calendar does it.

* Overhaul all the RSS feeds in Moodle to make use of a long hash-like string in the URL for identification (Forums, Data etc). We already have a mechanism for managing these, developed for the grade export reports, called user_key_login.
* Add RSS to other areas of Moodle such as the participants "last logins" and the activity logs.
* Explore/research other methods of opening up RSS in a safe way to the outside world.

'''Mentor:''' [http://moodle.org/user/view.php?id=240338&course=5 Nicolas Connault]

===New customisable theme===

Creation of a new [[Themes|theme]] that includes a configuration page so that the theme can be easily edited directly via Moodle. The page would include:

* Select a logo from the site files and/or interface to upload a file
* Several input fields to customise various CSS elements such as block headings, text colour, background colour, headings, header, footer, extra custom CSS etc.
* Select a parent theme

Also include the option of an administrator setting the theme for the site with their particular configuration or allowing users to customise various elements of the theme.

'''Mentor:''' [http://moodle.org/user/view.php?id=11995 Shane Elliott]

==Ideas we're considering for GSOC==

''Please add your ideas for GSOC projects under this heading.''

Following a review, approved suggested projects which have mentors will be moved under the heading "GSOC suggested projects" above. Please note that suggested projects require a mentor in order for them to be approved.

=== Roles interface improvements ===

Improve the roles editing interface by making it more dynamic and flexible. You can use the YUI library for ajax if you like, but there must be a good fall-back interface too. Some ideas include:

* Improve the order of the capabilities and implement better grouping.
* Make the groups of capabilities collapsible to make it easier to "zoom in" to a particular section.
* Add floating tooltip help when you hover over any given capability.
* Experiment with an (alternate) interface seeing the roles all side-by-side for easier comparison/editing.
* Implement a roles backup and restore system to allow site administrators to distribute useful roles in the Moodle community (rather than having to describe the permissions setup)
* More clearly designate the scope of capabilities. Does the capability apply at the site, category, course, activity, or user to user levels.
* Analyse the problem carefully with feedback from the community for more ideas.

===Integration with bibliographic systems such as Wikindx===

Managing references and citing them is an important behaviour in university education and research. Bibliographic facilities are quite complicated and go beyond the capabilities of Moodle built-in technology (e.g. the database activity). Integrating Moodle with open-source bibliographic software such as [http://wikindx.sourceforge.net/ Wikindx] could much facilitate this practice within Moodle.

Design and construct an integration with Wikindx (or other open-source bibliographic tools, if appropriate).

Teachers should be able to easily refer to wikindx bibliography items throughout a Moodle course, and be able to:

* Generate correctly-formatted in-place references (using standard styles e.g. Harvard, APA) for the commonly-cited reference types (e.g. journal article, book chapter, book). It may be possible to delegate the formatting directly to wikindx (since it already performs functions like these) rather than implementing a whole new set of logic in the Moodle integration.
** Possibly, allow some kind of direct searching of a wikindx database from within Moodle, so as to make it easier to refer to items
* Generate reading lists / bibliographies
* Allow export of the above into common machine-readable formats such as Bibtex or RIS. (Wikindx can perform this so again it's a question of hooking into, or expanding, wikindx functionality.)

See also [[Development:Wikindx]] and [http://moodle.org/mod/forum/discuss.php?d=23022 this forum discussion] too

===Implement CATs in Moodle===

[http://www.amazon.com/Classroom-Assessment-Techniques-Handbook-Education/dp/1555425003/ This book] describes a number of assessment techniques that involve collaboration and group work, and therefore fit very well into Moodle's Social Constructivist philosophy. The book about using these techniques in the classroom, but at an conference I attended there was a talk by Jean Runyon and Thomas Gorecki from the College of Southern Maryland saying how well these techniques work online.

Some of them just need a forum of something basic, but others would need to be done as a database module templates. Doing this would

# Make Moodle an even better teaching tool.
# Provide some good exemplars of how to do interesting things with database templates.
# Possibly highlight limitations in the database template system, that would require improvements to the database module code to overcome.
# One of the outcomes of this project could be some really good 'How to write a database template' tutorial, which would be very valuable to the community.

This is my (Tim Hunt's) idea, but I would not be interested in mentoring it. It would need to be someone who knows all about the database module.

=== Blog Assignment Type ===

'''What is it for?'''

Blogs are inherently user owned and driven by definition, however teachers would like to have a way to use blogs as assignments in their courses, comment, grade, etc.

A solution to this conundrum would be a new Assignment type that makes it possible for a student to easily submit the link one of their blogs to an assignment in a course, and have that blog entry privately graded and commented on by the instructor of that course. The blog entry itself remains public along with all of the student's other blogs. The tool would provide a time filter in the Assignment, so that the instructor could limit the blog entry posting dates that will be accepted (both to ensure 'fresh' entries and to keep the list of entries the student chooses short for frequent student bloggers).

I suggest a blog assignment type that functions as follows:

'''Teacher:'''
Teacher creates a blog assignment, this puts a grade in the gradebook and provides an assignment link for students to see.

The teacher can configure options to allow a blog entry posted in the last X days to be used for the assignment.

'''Students:'''
When students enter the assignment they see a drop down menu of all their current (blog entries posted in the last X days as chosen by the teacher) blog entries, and they choose one to satisfy the assignment.

'''Assessment:'''
The teacher goes in to the Assignment, views the blogs that are linked from the assignment, leaves comments (the comments only show up to the student when they view the blog assignment), and gives a grade.

The blog entries themselves stay on the student's blog, only the grades, links, and comments go in the assignment (optionally the assignment scrapes the student's blog and loads the full text in to store for backup and restore).

'''Interested Student''': [http://moodle.org/user/view.php?id=27192&course=1 Joey Morwick]

=== Moodle IDE ===
The objective of this project is to create a Moodle IDE based on Eclipse so that new developers can get to develop Moodle in less time.

'''Ideas'''
* First release: Eclipse + Plugins + Moodle splashscreen + CVS Configuration + Apache + MySQL + Moodle CVS code. All preconfigured and ready out of the box.
* Every X time, it checks for new versions of Moodle and packs it.
* Checkstyle plugin alike to follow Moodle Design Guidelines.
* Development help (can be wiki pages to navigate offline)

'''More mentors needed'''
'''Mentor''': [http://moodle.org/user/view.php?id=153093&course=1 David Horat]

'''Download (Old Version)''': [http://mirrors.davidhorat.com/moodle/MoodleIDE-latest.zip MoodleIDE-latest]

'''Join the [http://moodle.org/mod/forum/discuss.php?d=92859 discussion thread]'''

=== Moodle Multisite ===

The objective of this project is to make Moodle able to manage several Moodle sites with just one source code. This will help administrators of several sites to centralize code upgrades and ''technical problems''.

'''Ideas'''
* config.php will be a proxy that executes the concrete config file depending on the current url.
* The concrete config file will be under the directory /config and will have an arbitrary name created by the user.
* Create an XML file with all the pairs URL(n)-ConfigFile(1).
* Every time this file changes, it should be created the config.php file cache (to speed up things).
* Develop a Web frontend to change this file, that can be just accessed by the super admin in every site.

'''Mentor''': [http://moodle.org/user/view.php?id=153093&course=1 David Horat]

'''Join the [http://moodle.org/mod/forum/discuss.php?d=92860 discussion thread]'''

=== Automatic Accessibility Checking ===

The objective of this project is to make a tool that can check Moodle´s accessibility automatically before realising a version.

'''Ideas'''
* Automatic Tool (see down)
* Create an example course which introduces every XHTML code of the Moodle application, in order to have a way to test it.
* Create a button called "Check XHTML" which will send the current page code to a validator.
** This button just helps to check if the current page is valid. Good for development purposes.
** This button will be (de)activated in the administration panel
** Keep in mind that this button will not pass the URL as a referer, but it will take all the code from the page and send it to the validation service
** This button can be used in other web applications

'''Automatic Tool'''
* Create (or use an existing one) an automatic and easy to use tool that retrieves all the pages in a site until a concrete level (recursive calling it for each link)
** Use concrete user agent names
** Be able to store cookies in order to check it with a concrete user and password
* Plug each output to an automatic XHTML validator
** In the future we can plug in different validators: CSS, WCAG (what it can be checkable automatically), etc.
* Generate stats

'''Mentor''': [http://moodle.org/user/view.php?id=153093&course=1 David Horat]

'''Join the [http://moodle.org/mod/forum/discuss.php?d=92861 discussion thread]'''

=== Moodle Usability Guidelines ===
The main objective of this project is to develop a usability guidelines and testing methods in order to have a real view of Moodle usage.

'''Main tasks'''
* Research in the usability world to see which methods are most used and which are most reliable.
* Select the ones that are more appropriate for Moodle
* Create a usability guideline with the way to apply these methods for Moodle
* Try them with real world people (optional, although it would be very interesting)
* Analyse the results in order to extract conclusions of the current Moodle Usage
* Report the ways we could improve current Moodle Usability

'''Mentor''': [http://moodle.org/user/view.php?id=153093&course=1 David Horat]

'''Join the [http://moodle.org/mod/forum/discuss.php?d=92862 discussion thread]'''

===Moodle Target tracking + rewards/gifts===

This one would suit a facebook enthusiast! The Facebook gifts system is very popular and an ideal motivator for students at school level. Moodle needs an equivalent.

The aim is to develop a system for setting targets e.g. 'get at least 60% in this assignment/quiz/whatever' and for reward credits to be awarded automatically when it is completed. These reward credits translate to a system of Gifts similar to Facebook ones (small images, akin to stickers, which can be bought for a small fee and sent to friends who then display them on their profile). Admins can control what the reward images are for their site, maybe having them in some sort of rank order of value, or having different ones with different meanings. There are several ways they could be implemented beyond that:
* Teachers could choose which rewards can be earned for a specific task. For example, there may be different rewards for forums and assignments
* Teachers may want to specify different rewards for different levels of performance on the same task
* Teachers decide how many points a certain target is worth and the students cash in those points for rewards of various values
* Teachers decide how many rewards a target is worth and students choose that many rewards. None are worth more than 1 point each, the only difference is the graphic.
Once students have rewards, they need a way of allocating them (presumably permanently) to their friends. A system for logically displaying them on the friend's profile page would be needed too and will need to take into account
* Students with only a few stickers given to them by friends will want them to maximise their display
* Students who are very popular with hundreds of stickers may need to groups of 10 similar ones substituted for a super-sticker to use less space
* Hovering over the sticker should give details of who awarded it and when and possibly a short message.
Additionally, the social rating/ranking that this achieves will have most effect when it is publicly visible in various places e.g. under forum avatars and maybe in the online users block as well as the user profile.

Maybe this would sit well with the 'Friends' part of last years GSOC social networking project if it were finished.

I can't offer to mentor in coding terms, but will help with ideas all I can. [[User:Matt Gibson|Matt Gibson]] 05:26, 4 March 2008 (CST)

Interested Student : Rajan Vaish ( GTALK/GMAIL:vaish.rajan@gmail.com, IRC: vaish)

===Use of Video Material Effectively within Moodle===

The main purpose of this module will be to get use of video content, most probably a lecture, in a more formal/controlled/effective way within Moodle.

The main features of this module will be;
* Stream the video file (most probably in flv format)
** Should be able to stream by using multiple ways (e.x. FMS, Red5, lighttpd, etc..)
** Develop a front-end to configure the streaming server settings
** automatic media conversion (controlled by other Moodle settings like max file size, etc..)
* The teacher can give related notes, websites or other resources with the video
* The students can keep a note related to each video
* Share/view other student's notes (probably controlled by the teachers and/or students)
* Optionally assess the student notes and include them in the grade book

The above list may expanded based on your views.

'''Difficulty Level:''' Easy/Medium

'''Skills Required:'''
* Familiarity with streaming servers like Red5 and video streaming in general.
* Should be able to install and configure required products (Moodle, Red5, lighttpd, etc..) in any OS (Windows, Linux, Mac, etc..)
* Understanding of UI design concepts.
* Familiarity with Moodle APIs and [[Developer_documentation |Developing Moodle in general]]

'''Join the [http://moodle.org/mod/forum/discuss.php?d=92310 discussion thread]'''

'''References'''
* [http://moodle.org/mod/data/view.php?rid=931&page=102 FlashVideo]
* [http://osflash.org/red5 RED5 Open Source Streaming Server]

'''Potential Mentor:''' [http://moodle.org/user/edit.php?id=68520 Rashan Anushka]

===Web-based upgrade and plugins interface===

A system for Moodle to be upgraded along with its plugins from within the admin interface.

Currently, adding third party modules/patches can be messy and can especially slow down the upgrade process if CVS is not used. The aim would be to develop a way to package plugins in a standardised way so that a single zip file could be retrieved and installed automatically, similar to the way courses are restored. An added feature would be for the modules and plugins database on moodle.org to have an XML index so that you could browse available plugins from within your own moodle instance and have a one-click install option. Version compatibility info could be included so that only those which will work with your install will be shown. At upgrade time, the new third-party components screen could then show which ones could be upgraded along with the rest of the code and which would need to dropped because there is no compatible replacement.

===Add SQLite Support to XMLDB===
Having local sqlite database has been discussed in the past by developers for uses such as prototyping and testing. In this project you would add sqllite to the moodle database abstraction layer. Once this had been archieved you would look at adding a mechanism to copy a live moodle database into a sqllite database to implement a mechnism to switch into 'testing mode' using a sqlite copy of the live database. See also [http://moodle.org/mod/forum/discuss.php?d=88827 this thread on sqllite in moodle]

==See also==

* Using Moodle [http://moodle.org/mod/forum/discuss.php?d=92797 Simple Quiz building user interface] forum discussion about work that might be done this summer, although as part of something other than GSOC

[[Category:Developer]]
[[Category:Project]]

Projects for new developers

2008-03-20T11:01:41Z

Poltawski: Added feed consumtion project

This page lists possible student projects in Moodle, in particular suggestions for GSOC 2008 projects.
__NOTOC__
===Potential GSOC students===

Firstly, review the [http://code.google.com/p/google-summer-of-code/wiki/AdviceforStudents GSOC advice for students].

You may choose a project from the list below and/or suggest your own project ideas. Please use the [http://moodle.org/mod/forum/view.php?id=7105 Moodle.org Student projects forum] for discussing project ideas.

Student applications start on Monday 24th March 2008. See the [http://code.google.com/opensource/gsoc/2008/faqs.html#0.1_timeline GSOC timeline] for more information.

===Potential mentors===

Moodle developers willing to mentor a GSOC project, please suggest a new project under the heading "Ideas we're considering for GSOC" and/or add your name to an existing project. Contact [http://moodle.org/user/view.php?id=24152&course=5 Helen Foster] if you have any questions.

__TOC__
==Useful links==

* [[GSOC]] - describing Moodle's involvement with Google in their Summer of Code program, including a list of GSOC projects from 2006 and 2007
* [http://tracker.moodle.org/secure/IssueNavigator.jspa?mode=hide&requestId=10512 Popular Requests] - Most voted-for enhancements and new feature requests in the tracker

==GSOC suggested projects==

''List of approved suggested projects coming soon!''

Following a review, approved suggested projects which have mentors will be moved under this heading.

==Ideas we're considering for GSOC==

''Please add your ideas for GSOC projects under this heading.''

Following a review, approved suggested projects which have mentors will be moved under the heading "GSOC suggested projects" above. Please note that suggested projects require a mentor in order for them to be approved.

=== Secure RSS feeds ===

Currently RSS is less than useful because:
* Either we can't publish private information to the outside world because it's too sensitive.
* We open up sensitive information to the outside world

We should add codes to make the RSS URLs practically impossible to guess, much the same way as Google Calendar does it.

* Overhaul all the RSS feeds in Moodle to make use of a long hash-like string in the URL for identification (Forums, Data etc).
* Store these codes in a new field per-user and per-course.
* Add GUIs to let the user recreate their code to something else should they suspect a breach.
* Add RSS to other areas of Moodle such as the participants "last logins" and the activity logs.
* Explore/research other methods of opening up RSS in a safe way to the outside world.

'''Old Specification:''' [[Student projects/Secure RSS feeds|Secure RSS feeds]]

=== Consuming Feeds ===

RSS/Atom feeds are becoming an important technology on the web and its crucial that Moodle has good support for consuming these feeds for a variety of different uses. At present we generate RSS feeds for use by other applications, but at only consume feeds in the RSS block. It would be useful to have a core library which can take care of aggreating feeds (and the issues around that) and provide them in a simple format for plugins and other core parts of Moodle to use.

Create a feed aggregation library which:
* (Is probably based on an existing php framework)
* Will consume most current aggregation technologies (RSS 0.90/1.0/2.0 /Atom etc)
* Will work behind a variety of web proxies
* Periodically aggregates and uses a caching mechanism (doesn't query remote server with every request)

In order test this library refactor the RSS block to use it.

'''Mentor:''' Dan Poltawski

=== Roles interface improvements ===

Improve the roles editing interface by making it more dynamic and flexible. You can use the YUI library for ajax if you like, but there must be a good fall-back interface too. Some ideas include:

* Improve the order of the capabilities and implement better grouping.
* Make the groups of capabilities collapsible to make it easier to "zoom in" to a particular section.
* Add floating tooltip help when you hover over any given capability.
* Experiment with an (alternate) interface seeing the roles all side-by-side for easier comparison/editing.
* Implement a roles backup and restore system to allow site administrators to distribute useful roles in the Moodle community (rather than having to describe the permissions setup)
* More clearly designate the scope of capabilities. Does the capability apply at the site, category, course, activity, or user to user levels.
* Analyse the problem carefully with feedback from the community for more ideas.

===Integration with bibliographic systems such as Wikindx===

Managing references and citing them is an important behaviour in university education and research. Bibliographic facilities are quite complicated and go beyond the capabilities of Moodle built-in technology (e.g. the database activity). Integrating Moodle with open-source bibliographic software such as [http://wikindx.sourceforge.net/ Wikindx] could much facilitate this practice within Moodle.

Design and construct an integration with Wikindx (or other open-source bibliographic tools, if appropriate).

Teachers should be able to easily refer to wikindx bibliography items throughout a Moodle course, and be able to:

* Generate correctly-formatted in-place references (using standard styles e.g. Harvard, APA) for the commonly-cited reference types (e.g. journal article, book chapter, book). It may be possible to delegate the formatting directly to wikindx (since it already performs functions like these) rather than implementing a whole new set of logic in the Moodle integration.
** Possibly, allow some kind of direct searching of a wikindx database from within Moodle, so as to make it easier to refer to items
* Generate reading lists / bibliographies
* Allow export of the above into common machine-readable formats such as Bibtex or RIS. (Wikindx can perform this so again it's a question of hooking into, or expanding, wikindx functionality.)

See also [[Development:Wikindx]] and [http://moodle.org/mod/forum/discuss.php?d=23022 this forum discussion] too

===Implement CATs in Moodle===

[http://www.amazon.com/Classroom-Assessment-Techniques-Handbook-Education/dp/1555425003/ This book] describes a number of assessment techniques that involve collaboration and group work, and therefore fit very well into Moodle's Social Constructivist philosophy. The book about using these techniques in the classroom, but at an conference I attended there was a talk by Jean Runyon and Thomas Gorecki from the College of Southern Maryland saying how well these techniques work online.

Some of them just need a forum of something basic, but others would need to be done as a database module templates. Doing this would

# Make Moodle an even better teaching tool.
# Provide some good exemplars of how to do interesting things with database templates.
# Possibly highlight limitations in the database template system, that would require improvements to the database module code to overcome.
# One of the outcomes of this project could be some really good 'How to write a database template' tutorial, which would be very valuable to the community.

This is my (Tim Hunt's) idea, but I would not be interested in mentoring it. It would need to be someone who knows all about the database module.

=== Blog Assignment Type ===

'''What is it for?'''

Blogs are inherently user owned and driven by definition, however teachers would like to have a way to use blogs as assignments in their courses, comment, grade, etc.

A solution to this conundrum would be a new Assignment type that makes it possible for a student to easily submit the link one of their blogs to an assignment in a course, and have that blog entry privately graded and commented on by the instructor of that course. The blog entry itself remains public along with all of the student's other blogs. The tool would provide a time filter in the Assignment, so that the instructor could limit the blog entry posting dates that will be accepted (both to ensure 'fresh' entries and to keep the list of entries the student chooses short for frequent student bloggers).

I suggest a blog assignment type that functions as follows:

'''Teacher:'''
Teacher creates a blog assignment, this puts a grade in the gradebook and provides an assignment link for students to see.

The teacher can configure options to allow a blog entry posted in the last X days to be used for the assignment.

'''Students:'''
When students enter the assignment they see a drop down menu of all their current (blog entries posted in the last X days as chosen by the teacher) blog entries, and they choose one to satisfy the assignment.

'''Assessment:'''
The teacher goes in to the Assignment, views the blogs that are linked from the assignment, leaves comments (the comments only show up to the student when they view the blog assignment), and gives a grade.

The blog entries themselves stay on the student's blog, only the grades, links, and comments go in the assignment (optionally the assignment scrapes the student's blog and loads the full text in to store for backup and restore).

=== Moodle IDE ===
The objective of this project is to create a Moodle IDE based on Eclipse so that new developers can get to develop Moodle in less time.

'''Ideas'''
* First release: Eclipse + Plugins + Moodle splashscreen + CVS Configuration + Apache + MySQL + Moodle CVS code. All preconfigured and ready out of the box.
* Every X time, it checks for new versions of Moodle and packs it.
* Checkstyle plugin alike to follow Moodle Design Guidelines.
* Development help (can be wiki pages to navigate offline)

'''More mentors needed'''
'''Mentor''': [http://moodle.org/user/view.php?id=153093&course=1 David Horat]

'''Download (Old Version)''': [http://mirrors.davidhorat.com/moodle/MoodleIDE-latest.zip MoodleIDE-latest]

'''Join the [http://moodle.org/mod/forum/discuss.php?d=92859 discussion thread]'''

=== Moodle Multisite ===

The objective of this project is to make Moodle able to manage several Moodle sites with just one source code. This will help administrators of several sites to centralize code upgrades and ''technical problems''.

'''Ideas'''
* config.php will be a proxy that executes the concrete config file depending on the current url.
* The concrete config file will be under the directory /config and will have an arbitrary name created by the user.
* Create an XML file with all the pairs URL(n)-ConfigFile(1).
* Every time this file changes, it should be created the config.php file cache (to speed up things).
* Develop a Web frontend to change this file, that can be just accessed by the super admin in every site.

'''Mentor''': [http://moodle.org/user/view.php?id=153093&course=1 David Horat]

'''Join the [http://moodle.org/mod/forum/discuss.php?d=92860 discussion thread]'''

=== Automatic Accessibility Checking ===

The objective of this project is to make a tool that can check Moodle´s accessibility automatically before realising a version.

'''Ideas'''
* Automatic Tool (see down)
* Create an example course which introduces every XHTML code of the Moodle application, in order to have a way to test it.
* Create a button called "Check XHTML" which will send the current page code to a validator.
** This button just helps to check if the current page is valid. Good for development purposes.
** This button will be (de)activated in the administration panel
** Keep in mind that this button will not pass the URL as a referer, but it will take all the code from the page and send it to the validation service
** This button can be used in other web applications

'''Automatic Tool'''
* Create (or use an existing one) an automatic and easy to use tool that retrieves all the pages in a site until a concrete level (recursive calling it for each link)
** Use concrete user agent names
** Be able to store cookies in order to check it with a concrete user and password
* Plug each output to an automatic XHTML validator
** In the future we can plug in different validators: CSS, WCAG (what it can be checkable automatically), etc.
* Generate stats

'''Mentor''': [http://moodle.org/user/view.php?id=153093&course=1 David Horat]

'''Join the [http://moodle.org/mod/forum/discuss.php?d=92861 discussion thread]'''

=== Moodle Usability Guidelines ===
The main objective of this project is to develop a usability guidelines and testing methods in order to have a real view of Moodle usage.

'''Main tasks'''
* Research in the usability world to see which methods are most used and which are most reliable.
* Select the ones that are more appropriate for Moodle
* Create a usability guideline with the way to apply these methods for Moodle
* Try them with real world people (optional, although it would be very interesting)
* Analyse the results in order to extract conclusions of the current Moodle Usage
* Report the ways we could improve current Moodle Usability

'''Mentor''': [http://moodle.org/user/view.php?id=153093&course=1 David Horat]

'''Join the [http://moodle.org/mod/forum/discuss.php?d=92862 discussion thread]'''

===Moodle Target tracking + rewards/gifts===

This one would suit a facebook enthusiast! The Facebook gifts system is very popular and an ideal motivator for students at school level. Moodle needs an equivalent.

The aim is to develop a system for setting targets e.g. 'get at least 60% in this assignment/quiz/whatever' and for reward credits to be awarded automatically when it is completed. These reward credits translate to a system of Gifts similar to Facebook ones (small images, akin to stickers, which can be bought for a small fee and sent to friends who then display them on their profile). Admins can control what the reward images are for their site, maybe having them in some sort of rank order of value, or having different ones with different meanings. There are several ways they could be implemented beyond that:
* Teachers could choose which rewards can be earned for a specific task. For example, there may be different rewards for forums and assignments
* Teachers may want to specify different rewards for different levels of performance on the same task
* Teachers decide how many points a certain target is worth and the students cash in those points for rewards of various values
* Teachers decide how many rewards a target is worth and students choose that many rewards. None are worth more than 1 point each, the only difference is the graphic.
Once students have rewards, they need a way of allocating them (presumably permanently) to their friends. A system for logically displaying them on the friend's profile page would be needed too and will need to take into account
* Students with only a few stickers given to them by friends will want them to maximise their display
* Students who are very popular with hundreds of stickers may need to groups of 10 similar ones substituted for a super-sticker to use less space
* Hovering over the sticker should give details of who awarded it and when and possibly a short message.
Additionally, the social rating/ranking that this achieves will have most effect when it is publicly visible in various places e.g. under forum avatars and maybe in the online users block as well as the user profile.

Maybe this would sit well with the 'Friends' part of last years GSOC social networking project if it were finished.

I can't offer to mentor in coding terms, but will help with ideas all I can. [[User:Matt Gibson|Matt Gibson]] 05:26, 4 March 2008 (CST)

Interested Student : Rajan Vaish ( GTALK/GMAIL:vaish.rajan@gmail.com, IRC: vaish)

===Use of Video Material Effectively within Moodle===

The main purpose of this module will be to get use of video content, most probably a lecture, in a more formal/controlled/effective way within Moodle.

The main features of this module will be;
* Stream the video file (most probably in flv format)
** Should be able to stream by using multiple ways (e.x. FMS, Red5, lighttpd, etc..)
** Develop a front-end to configure the streaming server settings
** automatic media conversion (controlled by other Moodle settings like max file size, etc..)
* The teacher can give related notes, websites or other resources with the video
* The students can keep a note related to each video
* Share/view other student's notes (probably controlled by the teachers and/or students)
* Optionally assess the student notes and include them in the grade book

The above list may expanded based on your views.

'''Difficulty Level:''' Easy/Medium

'''Skills Required:'''
* Familiarity with streaming servers like Red5 and video streaming in general.
* Should be able to install and configure required products (Moodle, Red5, lighttpd, etc..) in any OS (Windows, Linux, Mac, etc..)
* Understanding of UI design concepts.
* Familiarity with Moodle APIs and [[Developer_documentation |Developing Moodle in general]]

'''Join the [http://moodle.org/mod/forum/discuss.php?d=92310 discussion thread]'''

'''References'''
* [http://moodle.org/mod/data/view.php?rid=931&page=102 FlashVideo]
* [http://osflash.org/red5 RED5 Open Source Streaming Server]

'''Potential Mentor:''' [http://moodle.org/user/edit.php?id=68520 Rashan Anushka]

===Web-based upgrade and plugins interface===

A system for Moodle to be upgraded along with its plugins from within the admin interface.

Currently, adding third party modules/patches can be messy and can especially slow down the upgrade process if CVS is not used. The aim would be to develop a way to package plugins in a standardised way so that a single zip file could be retrieved and installed automatically, similar to the way courses are restored. An added feature would be for the modules and plugins database on moodle.org to have an XML index so that you could browse available plugins from within your own moodle instance and have a one-click install option. Version compatibility info could be included so that only those which will work with your install will be shown. At upgrade time, the new third-party components screen could then show which ones could be upgraded along with the rest of the code and which would need to dropped because there is no compatible replacement.

==See also==

* Using Moodle [http://moodle.org/mod/forum/discuss.php?d=92797 Simple Quiz building user interface] forum discussion about work that might be done this summer, although as part of something other than GSOC

[[Category:Developer]]
[[Category:Project]]