View source for Tutorial on using git in Moodle development

{{obsolete}}
This page is not valid any more. It describes how developers could use Git when Moodle's main repository was CVS. See [[Git for developers]] for up-to-date documentation.

'''Disclaimer:''' This is not a tutorial on git itself. You are supposed to know the basic concepts of git (notably cloning, branching and merging). This tutorial is based on my personal experience with using git in Moodle development process. There are other (and almost certainly better) ways of doing it. See the references at the bottom of the page. Quite often, I found a solution that just worked and did not explore alternatives. You are warmly welcome to contribute and improve this tutorial. Please use the talk page to add/fix something so I can merge it into the tutorial text. --[[User:David Mudrak|David Mudrak]]


== Preparing the environment ==

[[Image:moodle-git-cvs-servers.png|frame|Set-up of servers used in this tutorial]]

In this tutorial, we will use an environment as illustrated in the diagram. There are currently two source code repositories you can use: the main Moodle CVS server cvs.moodle.org and its git mirror at git.moodle.org. While the CVS server contains both Moodle core code and contributed code, the git server mirrors the core code only (this may be changed in the near future). We expect you have a server (which we will call "nostromo") that you and your team members use and that you do the actual developing on localhost machines.

Later in this tutorial, you will see how
* <syntaxhighlight lang="php">git fetch</syntaxhighlight> can be used to get the recent changes from upstream to nostromo
* <syntaxhighlight lang="php">git cvsimport</syntaxhighlight> can be used to create your own git mirror of contributed code (we do not do this yet for you)
* <syntaxhighlight lang="php">git clone</syntaxhighlight> can be used to create a mirror of your nostromo repos on you local machines
* <syntaxhighlight lang="php">git push</syntaxhighlight> can be used to backup your own work and to share it with your friend
* <syntaxhighlight lang="php">git pull</syntaxhighlight> can be used regularly to fetch and merge the work done by your friend
* <syntaxhighlight lang="php">git cvsexportcommit</syntaxhighlight> can be used to send your commits upstream (and you will, right? ;-)

== Create your own Moodle source code mirror ==

Make sure you and your team friends have write access to <syntaxhighlight lang="php">/pub/scm/git</syntaxhighlight> at your nostromo server.

 $ ssh nostromo
 $ cd /pub/scm/git
 $ mkdir moodle.git
 $ cd moodle.git
 $ git init --bare --shared=group

That will create an empty bare repository in /pub/scm/git/moodle.git directory. Shortly, the "bare" means it contains just the repository itself, not the working copy (checkout) of the files.

 $ git --bare fetch git://git.moodle.org/moodle.git cvshead:cvshead
 $ git --bare fetch git://git.moodle.org/moodle.git MOODLE_19_STABLE:MOODLE_19_STABLE
 $ git --bare fetch git://git.moodle.org/moodle.git MOODLE_18_STABLE:MOODLE_18_STABLE

These three commands will create three branches - cvshead, MOODLE_19_STABLE and MOODLE_18_STABLE from the upstream and will fetch all the Moodle development history. You can re-run the same commands later to fetch the recent changes. They can be run by cron to regularly fetch the latest changes from the upstream to your nostromo repository. It makes sense to run them every 30 minutes as it is the interval of the git.moodle.org updates. 

== Check out the working copies of the repository ==

Now let us go to your localhost machine and check out the working copy from the nostromo repository

 $ cd ~/public_html
 $ git clone ssh+git://nostromo/pub/scm/git/moodle.git moodle19
 $ cd moodle19
 $ git checkout -b MOODLE_19_STABLE origin/MOODLE_19_STABLE

After these commands, you have a checkout of the latest 1.9.x Moodle branch in your <syntaxhighlight lang="php">~/public_html/moodle19</syntaxhighlight>. To update it (it est to fetch the changes from your nostromo and to merge them), just run

 $ git pull

== Prepare a customized branch for your client ==

Imagine your are going to install and customize Moodle 1.9 for a client, for example the Hogwarts School. Let us create a branch for them where the whole customization will be done.

 $ cd ~/public_html/moodle19
 $ git checkout -b mdl19-hogwarts origin/MOODLE_19_STABLE
 $ ... (do all your customization here, git add, git commit etc)
 
Now, publish your changes to nostromo so your friend can clone it. Also, pushing your stuff to a server is quite a good backup of your work.

 $ git push origin mdl19-hogwarts:mdl19-hogwarts

Later on, just <syntaxhighlight lang="php">git push</syntaxhighlight> should be enough as the remote branch already exists but this depends on your git settings.

Your friend can make their own clone of your customizations with

 $ cd ~/public_html/moodle19
 $ git checkout -b mdl19-hogwarts origin/mdl19-hogwarts
 $ ... (hack something, git add, git commit)
 $ git push

== Deploying a customized branch ==

=== With a shell access to the client's webhost ===

If you can, install git at the client's webhost. Then you can just create another clone of your customer's branch using the same commands as when you did during the checking out to your development machine.

=== With FTP access only ===

 $ cd ~/public_html/moodle19
 $ git checkout mdl19-hogwarts
 $ git pull
 $ git tag mdl19-hogwarts-20091026
 $ git archive --format=tar --prefix=hogwarts/htdocs/ mdl19-hogwarts-20091026 | (cd /tmp/ && tar xf -)

Now you can upload <syntaxhighlight lang="php">/tmp/hogwarts/htdocs/</syntaxhighlight> to the server.

=== Uploading just the changed files ===

 $ cd ~/public_html/moodle19
 $ git checkout mdl19-hogwarts
 $ git pull
 $ git tag mdl19-hogwarts-20100129
 $ mkdir /tmp/upload
 $ cp --parents `git diff --name-only mdl19-hogwarts-20091026..mdl19-hogwarts-20100129` /tmp/upload/

In <syntaxhighlight lang="php">/tmp/upload/</syntaxhighlight>, you will find just the files that were modified during the given period (tags used here). Then you can upload just those files to the server.

== Sending a single patch upstream ==

To be able to export patch from git and to commit it into CVS, you will need two Moodle checkouts - the git one and the CVS one. Let us say you have a local git clone in <syntaxhighlight lang="php">~/public_html/moodlehead</syntaxhighlight> and a pure CVS checkout of HEAD at <syntaxhighlight lang="php">~/cvs/moodlehead</syntaxhighlight>. Firstly, let us prepare some helpers.

 $ cd ~/public_html/moodlehead
 $ git config cvsexportcommit.cvsdir ~/cvs/moodlehead
 $ git config --global alias.cvsupstream 'cvsexportcommit -c -p -u -v'

We just said git where to look for the CVS version of the repository (so it is not needed to type it every time) and created an alias for cvsexportcommit command with some parameters (RTFM). Now, let us go to fix and commit a bug:

 $ git checkout -b somenastybug
 $ ... (edit, git add, git commit - this leads to commit eg 3d9779d
 $ git cvsupstream 3d9779d

The git commit will be re-applied to the CVS working copy and then committed to cvs.moodle.org as if you did it manually. It will be mirrored to git.moodle.org, fetched by nostromo and pulled back to your machine. When that happens, the command

 $ git cherry cvshead

shall inform you that all commits from the current branch are already included in cvshead - it est, there are no cherries. So, you can delete the local bug branch:
 
 $ git branch -D somenastybug

Please note, the commit id of your patch will be different in your fix branch and in the branch coming from CVS. That is why you need to use <syntaxhighlight lang="php">git cherry</syntaxhighlight> which compares the content of the patch.

== Sending whole branch into CVS ==

Imagine you start to implement some new feature or you are rewriting/extending some subsystem. Let us say you decide to replace the underlying QuickForms library with a new and better framework (yay! kudos for you then :-)). As it is common with git, you will branch, rebase and merge as crazy, ending with a local git branch (eg "new-mforms") containing a lot of commits. You will make sure that your changes clearly applies to the most recent upstream version (git is really helpful here) and now you want to send your branch upstream.

Of course, you could prepare just a huge patch a commit it separately. You are discouraged to do so, though. There are some good reasons why your work should be re-applied commit-by-commit to CVS.

 $ cd ~/public_html/moodlehead
 $ git fetch
 $ git checkout new-mforms
 $ git cherry

This will show a list of commits on your local branch with the information whether they are or are not in the remote tracking branch, too. You are interested in commits with "+" prefix, ie. those that are added on your branch only. Use the following command to iterate through such commits and commit them one-by-one upstream:

 $ git cherry origin/cvshead new-mforms | sed -n 's/^+ //p' | xargs -l1 git cvsupstream

== Branching a plugin from CONTRIB ==

We do not have a git mirror of our CONTRIB area yet so you will have to create it yourself. Let us say you need to customize the favourite [[Certificate module]] for your client.

 $ ssh nostromo
 $ cd /pub/scm/git
 $ mkdir mod-certificate

Create an executable bash script /pub/scm/git/cvsimport-mod-certificate with the following contents (modify as needed)

 #!/bin/bash
 CVSROOT=:pserver:anonymous@uk.cvs.moodle.org:/cvsroot/moodle
 MODULE=contrib/plugins/mod/certificate
 INSTALLDIR=mod-certificate
 LOGFILE=mod-certificate.log
 git cvsimport -p x -v -k -o cvshead -d $CVSROOT -C $INSTALLDIR $MODULE &> $LOGFILE

Run this script at nostromo for the first time to fetch the whole history of the Certificate module. Then you can run it regularly (every hour or so shall be enough) to keep your mirror up-to-date. Now let us create a working copy of this contrib module. At your localhost:

 $ cd ~/public_html/moodle19/mod
 $ git clone ssh+git://nostromo/pub/scm/git/mod-certificate certificate
 $ cd certificate
 $ git checkout -b MOODLE_19_STABLE origin/MOODLE_19_STABLE

In order to ignore the new directory in your working area, add the line <syntaxhighlight lang="php">/mod/certificate</syntaxhighlight> to either <syntaxhighlight lang="php">.gitignore</syntaxhighlight> or to <syntaxhighlight lang="php">.git/info/exclude</syntaxhighlight> (I prefer the later)

 $ cd ~/public_html/moodle19
 $ echo /mod/certificate >> .git/info/exclude

Do not forget you have to manually include the certificate in all deployment processes as discussed above. As it is not part of your moodle19 checkout, it will not be pushed neither archived etc. If I understand it correctly, git submodules feature can be handy here but I haven't played with it yet.

== Deploying a customized branch with submodules ==

Situation: you have your own branch based on MOODLE_19_STABLE with some local tweaks. Inside it, you have additional separate git repos - one for the client's theme and for the customized mod/certificate/ module. Then you may find the following Makefile saved in the Moodle dirroot useful:

 EXPORT_TARGET=/tmp/moodle.client.tld/
 EXPORT_TAG=deploy/mdl19-client-20091030
 TAG_MSG="Upgrade to the recent 1.9.x"
 tag:
    git tag -m ${TAG_MSG} ${EXPORT_TAG}
    cd mod/certificate/ && git tag -m ${TAG_MSG} ${EXPORT_TAG}
    cd theme/client/ && git tag -m ${TAG_MSG} ${EXPORT_TAG}
 export:
    rm -rf ${EXPORT_TARGET}
    mkdir -p ${EXPORT_TARGET}
    git archive --format=tar --prefix=moodle/ ${EXPORT_TAG} | (cd ${EXPORT_TARGET} && tar xf -)
    (cd mod/certificate/ && git archive --format=tar --prefix=moodle/mod/certificate/ ${EXPORT_TAG}) | (cd ${EXPORT_TARGET} && tar xf -)
    (cd theme/client/ && git archive --format=tar --prefix=moodle/theme/client/ ${EXPORT_TAG}) | (cd ${EXPORT_TARGET} && tar xf -)

Then, running

 $ make tag
 $ make export

will create a snapshot in /tmp/moodle.client.tld/ to be deployed.

== Credits ==

A big thank you to Penny, Dan and Nigel for helping me with my first steps with git. Kudos for Martín and others from Catalyst for their contribution to the git-cvs integration tools.

== See also ==
* [[Quick_GIT_starting_guide_for_Moodle_development]]
* [[Tracking Moodle CVS with git]]
* [[Git tips]]
* [[Using GIT to backup moodledata]]
* [[Git Migration Workflows]]

[[Category:Git]]