Difference between revisions of "Obsolete:Tutorial on using git in Moodle development"

Jump to: navigation, search

Note: You are currently viewing documentation for Moodle 1.9. Up-to-date documentation for the latest stable version is available here: Tutorial on using git in Moodle development.

m (Preparing the environment: small rewording)
m (formatting)
Line 1: Line 1:
 
'''Disclaimer:''' This is not a tutorial on git itself. You are supposed to know the basic concepts of git (notably cloning, branching and merging). This tutorial is based on my personal experience with using git in Moodle development process. There are other (and almost certainly better) ways of doing it. See the references at the bottom of the page. Quite often, I found a solution that just worked and did not explore alternatives. You are warmly welcome to contribute and improve this tutorial. Please use the talk page to add/fix something so I can merge it into the tutorial text. --[[User:David Mudrak|David Mudrak]]
 
'''Disclaimer:''' This is not a tutorial on git itself. You are supposed to know the basic concepts of git (notably cloning, branching and merging). This tutorial is based on my personal experience with using git in Moodle development process. There are other (and almost certainly better) ways of doing it. See the references at the bottom of the page. Quite often, I found a solution that just worked and did not explore alternatives. You are warmly welcome to contribute and improve this tutorial. Please use the talk page to add/fix something so I can merge it into the tutorial text. --[[User:David Mudrak|David Mudrak]]
  
= Preparing the environment =
+
 
 +
== Preparing the environment ==
  
 
[[Image:moodle-git-cvs-servers.png|frame|Set-up of servers used in this tutorial]]
 
[[Image:moodle-git-cvs-servers.png|frame|Set-up of servers used in this tutorial]]
Line 15: Line 16:
 
* <code>git cvsexportcommit</code> can be used to send your commits upstream (and you will, right? ;-)
 
* <code>git cvsexportcommit</code> can be used to send your commits upstream (and you will, right? ;-)
  
= Create your own Moodle source codes mirror =
+
== Create your own Moodle source code mirror ==
  
 
Make sure you and your team friends have write access to <code>/pub/scm/git</code> at your nostromo server.
 
Make sure you and your team friends have write access to <code>/pub/scm/git</code> at your nostromo server.
Line 33: Line 34:
 
These three commands will create three branches - cvshead, MOODLE_19_STABLE and MOODLE_18_STABLE from the upstream and will fetch all the Moodle development history. You can re-run the same commands later to fetch the recent changes. They can be run by cron to regularly fetch the latest changes from the upstream to your nostromo repository. It makes sense to run theme every 30 minutes as it is the interval of the git.moodle.org updates.  
 
These three commands will create three branches - cvshead, MOODLE_19_STABLE and MOODLE_18_STABLE from the upstream and will fetch all the Moodle development history. You can re-run the same commands later to fetch the recent changes. They can be run by cron to regularly fetch the latest changes from the upstream to your nostromo repository. It makes sense to run theme every 30 minutes as it is the interval of the git.moodle.org updates.  
  
= Check out the working copies of the repository =
+
== Check out the working copies of the repository ==
  
 
Now let us go to your localhost machine and check out the working copy from the nostromo repository
 
Now let us go to your localhost machine and check out the working copy from the nostromo repository
Line 46: Line 47:
 
  $ git pull
 
  $ git pull
  
= Prepare a customized branch for your client =
+
== Prepare a customized branch for your client ==
  
 
Imagine your are going to install and customize Moodle 1.9 for a client, for example the Hogwarts School. Let us create a branch for them where the whole customization will be done.
 
Imagine your are going to install and customize Moodle 1.9 for a client, for example the Hogwarts School. Let us create a branch for them where the whole customization will be done.
Line 67: Line 68:
 
  $ git push
 
  $ git push
  
= Deploying a customized branch =
+
== Deploying a customized branch ==
  
== With a shell access to the client's webhost ==
+
=== With a shell access to the client's webhost ===
  
 
If you can, install git at the client's webhost. Then you can just create another clone of your customer's branch using the same commands as when you did during the checking out to your development machine.
 
If you can, install git at the client's webhost. Then you can just create another clone of your customer's branch using the same commands as when you did during the checking out to your development machine.
  
== With FTP access only ==
+
=== With FTP access only ===
  
 
  $ cd ~/public_html/moodle19
 
  $ cd ~/public_html/moodle19
Line 83: Line 84:
 
Now you can upload <code>/tmp/hogwarts/htdocs/</code> to the server
 
Now you can upload <code>/tmp/hogwarts/htdocs/</code> to the server
  
= Sending a single patch upstream =
+
== Sending a single patch upstream ==
  
 
To be able to export patch from git and to commit it into CVS, you will need two Moodle checkouts - the git one and the CVS one. Let us say you have a local git clone in <code>~/public_html/moodlehead</code> and a pure CVS checkout of HEAD at <code>~/cvs/moodlehead</code>. Firstly, let us prepare some helpers.
 
To be able to export patch from git and to commit it into CVS, you will need two Moodle checkouts - the git one and the CVS one. Let us say you have a local git clone in <code>~/public_html/moodlehead</code> and a pure CVS checkout of HEAD at <code>~/cvs/moodlehead</code>. Firstly, let us prepare some helpers.
Line 107: Line 108:
 
Please note, the commit id of your patch will be different in your fix branch and in the branch coming from CVS. That is why you need to use <code>git cherry</code> which compares the content of the patch.
 
Please note, the commit id of your patch will be different in your fix branch and in the branch coming from CVS. That is why you need to use <code>git cherry</code> which compares the content of the patch.
  
= Sending whole branch into CVS =
+
== Sending whole branch into CVS ==
  
 
Imagine you start to implement some new feature or you are rewriting/extending some subsystem. Let us say you decide to replace the underlying QuickForms library with a new and better framework (yay! kudos for you then :-)). As it is common with git, you will branch, rebase and merge as crazy, ending with a local git branch (eg "new-mforms") containing a lot of commits. You will make sure that your changes clearly applies to the most recent upstream version (git is really helpful here) and now you want to send your branch upstream.
 
Imagine you start to implement some new feature or you are rewriting/extending some subsystem. Let us say you decide to replace the underlying QuickForms library with a new and better framework (yay! kudos for you then :-)). As it is common with git, you will branch, rebase and merge as crazy, ending with a local git branch (eg "new-mforms") containing a lot of commits. You will make sure that your changes clearly applies to the most recent upstream version (git is really helpful here) and now you want to send your branch upstream.
Line 122: Line 123:
 
  $ git cherry origin/cvshead new-mforms | sed -n 's/^+ //p' | xargs -l1 git cvsupstream
 
  $ git cherry origin/cvshead new-mforms | sed -n 's/^+ //p' | xargs -l1 git cvsupstream
  
= Branching a plugin from CONTRIB =
+
== Branching a plugin from CONTRIB ==
  
 
We do not have a git mirror of our CONTRIB area yet so you will have to create it yourself. Let us say you need to customize the favourite [[Certificate module]] for your client.
 
We do not have a git mirror of our CONTRIB area yet so you will have to create it yourself. Let us say you need to customize the favourite [[Certificate module]] for your client.
Line 152: Line 153:
  
 
Do not forget you have to manually include the certificate in all deployment processes as discussed above. As it is not part of your moodle19 checkout, it will not be pushed neither archived etc. If I understand it correctly, git submodules feature can be handy here but I haven't played with it yet.
 
Do not forget you have to manually include the certificate in all deployment processes as discussed above. As it is not part of your moodle19 checkout, it will not be pushed neither archived etc. If I understand it correctly, git submodules feature can be handy here but I haven't played with it yet.
 
= Appendix =
 
  
 
== Credits ==
 
== Credits ==

Revision as of 11:36, 29 October 2009

Disclaimer: This is not a tutorial on git itself. You are supposed to know the basic concepts of git (notably cloning, branching and merging). This tutorial is based on my personal experience with using git in Moodle development process. There are other (and almost certainly better) ways of doing it. See the references at the bottom of the page. Quite often, I found a solution that just worked and did not explore alternatives. You are warmly welcome to contribute and improve this tutorial. Please use the talk page to add/fix something so I can merge it into the tutorial text. --David Mudrak


Preparing the environment

Set-up of servers used in this tutorial

In this tutorial, we will use an environment as illustrated in the diagram. There are currently two source code repositories you can use: the main Moodle CVS server cvs.moodle.org and its git mirror at git.moodle.org. While the CVS server contains both Moodle core code and contributed code, the git server mirrors the core code only (this may be changed in the near future). We expect you have a server (which we will call "nostromo") that you and your team members use and that you do the actual developing on localhost machines.

Later in this tutorial, you will see how

  • git fetch
    can be used to get the recent changes from upstream to nostromo
  • git cvsimport
    can be used to create your own git mirror of contributed code (we do not do this yet for you)
  • git clone
    can be used to create a mirror of your nostromo repos on you local machines
  • git push
    can be used to backup your own work and to share it with your friend
  • git pull
    can be used regularly to fetch and merge the work done by your friend
  • git cvsexportcommit
    can be used to send your commits upstream (and you will, right? ;-)

Create your own Moodle source code mirror

Make sure you and your team friends have write access to
/pub/scm/git
at your nostromo server.
$ ssh nostromo
$ cd /pub/scm/git
$ mkdir moodle.git
$ cd moodle.git
$ git init --bare --shared=group

That will create an empty bare repository in /pub/scm/git/moodle.git directory. Shortly, the "bare" means it contains just the repository itself, not the working copy (checkout) of the files.

$ git --bare fetch git://git.moodle.org/moodle.git cvshead:cvshead
$ git --bare fetch git://git.moodle.org/moodle.git MOODLE_19_STABLE:MOODLE_19_STABLE
$ git --bare fetch git://git.moodle.org/moodle.git MOODLE_18_STABLE:MOODLE_18_STABLE

These three commands will create three branches - cvshead, MOODLE_19_STABLE and MOODLE_18_STABLE from the upstream and will fetch all the Moodle development history. You can re-run the same commands later to fetch the recent changes. They can be run by cron to regularly fetch the latest changes from the upstream to your nostromo repository. It makes sense to run theme every 30 minutes as it is the interval of the git.moodle.org updates.

Check out the working copies of the repository

Now let us go to your localhost machine and check out the working copy from the nostromo repository

$ cd ~/public_html
$ git clone ssh+git://nostromo/pub/scm/git/moodle.git moodle19
$ cd moodle19
$ git checkout -b MOODLE_19_STABLE origin/MOODLE_19_STABLE
After these commands, you have a checkout of the latest 1.9.x Moodle branch in your
~/public_html/moodle19
. To update it (it est to fetch the changes from your nostromo and to merge them), just run
$ git pull

Prepare a customized branch for your client

Imagine your are going to install and customize Moodle 1.9 for a client, for example the Hogwarts School. Let us create a branch for them where the whole customization will be done.

$ cd ~/public_html/moodle19
$ git checkout -b mdl19-hogwarts origin/MOODLE_19_STABLE
$ ... (do all your customization here, git add, git commit etc)

Now, publish your changes to nostromo so your friend can clone it. Also, pushing your stuff to a server is quite a good backup of your work.

$ git push origin mdl19-hogwarts:mdl19-hogwarts
Later on, just
git push
should be enough as the remote branch already exists but this depends on your git settings.

Your friend can make their own clone of your customizations with

$ cd ~/public_html/moodle19
$ git checkout -b mdl19-hogwarts origin/mdl19-hogwarts
$ ... (hack something, git add, git commit)
$ git push

Deploying a customized branch

With a shell access to the client's webhost

If you can, install git at the client's webhost. Then you can just create another clone of your customer's branch using the same commands as when you did during the checking out to your development machine.

With FTP access only

$ cd ~/public_html/moodle19
$ git checkout mdl19-hogwarts
$ git pull
$ git tag mdl19-hogwarts-20091026
$ git archive --format=tar --prefix=hogwarts/htdocs/ mdl19-hogwarts-20091026 | (cd /tmp/ && tar xf -)
Now you can upload
/tmp/hogwarts/htdocs/
to the server

Sending a single patch upstream

To be able to export patch from git and to commit it into CVS, you will need two Moodle checkouts - the git one and the CVS one. Let us say you have a local git clone in
~/public_html/moodlehead
and a pure CVS checkout of HEAD at
~/cvs/moodlehead
. Firstly, let us prepare some helpers.
$ cd ~/public_html/moodlehead
$ git config cvsexportcommit.cvsdir ~/cvs/moodlehead
$ git config --global alias.cvsupstream 'cvsexportcommit -c -p -u -v'

We just said git where to look for the CVS version of the repository (so it is not needed to type it every time) and created an alias for cvsexportcommit command with some parameters (RTFM). Now, let us go to fix and commit a bug:

$ git checkout -b somenastybug
$ ... (edit, git add, git commit - this leads to commit eg 3d9779d
$ git cvsupstream 3d9779d

The git commit will be re-applied to the CVS working copy and then committed to cvs.moodle.org as if you did it manually. It will be mirrored to git.moodle.org, fetched by nostromo and pulled back to your machine. When that happens, the command

$ git cherry cvshead

shall inform you that all commits from the current branch are already included in cvshead - it est, there are no cherries. So, you can delete the local bug branch:

$ git branch -D somenastybug
Please note, the commit id of your patch will be different in your fix branch and in the branch coming from CVS. That is why you need to use
git cherry
which compares the content of the patch.

Sending whole branch into CVS

Imagine you start to implement some new feature or you are rewriting/extending some subsystem. Let us say you decide to replace the underlying QuickForms library with a new and better framework (yay! kudos for you then :-)). As it is common with git, you will branch, rebase and merge as crazy, ending with a local git branch (eg "new-mforms") containing a lot of commits. You will make sure that your changes clearly applies to the most recent upstream version (git is really helpful here) and now you want to send your branch upstream.

Of course, you could prepare just a huge patch a commit it separately. You are discouraged to do so, though. There are some good reasons why your work should be re-applied commit-by-commit to CVS.

$ cd ~/public_html/moodlehead
$ git fetch
$ git checkout new-mforms
$ git cherry

This will show a list of commits on your local branch with the information whether they are or are not in the remote tracking branch, too. You are interested in commits with "+" prefix, ie. those that are added on your branch only. Use the following command to iterate through such commits and commit them one-by-one upstream:

$ git cherry origin/cvshead new-mforms | sed -n 's/^+ //p' | xargs -l1 git cvsupstream

Branching a plugin from CONTRIB

We do not have a git mirror of our CONTRIB area yet so you will have to create it yourself. Let us say you need to customize the favourite Certificate module for your client.

$ ssh nostromo
$ cd /pub/scm/git
$ mkdir mod-certificate

Create an executable bash script /pub/scm/git/cvsimport-mod-certificate with the following contents (modify as needed)

#!/bin/bash
CVSROOT=:pserver:anonymous@uk.cvs.moodle.org:/cvsroot/moodle
MODULE=contrib/plugins/mod/certificate
INSTALLDIR=mod-certificate
LOGFILE=mod-certificate.log
git cvsimport -p x -v -k -o cvshead -d $CVSROOT -C $INSTALLDIR $MODULE &> $LOGFILE

Run this script at nostromo for the first time to fetch the whole history of the Certificate module. Then you can run it regularly (every hour or so shall be enough) to keep your mirror up-to-date. Now let us create a working copy of this contrib module. At your localhost:

$ cd ~/public_html/moodle19/mod
$ git clone ssh+git://nostromo/pub/scm/git/mod-certificate certificate
$ cd certificate
$ git checkout -b MOODLE_19_STABLE origin/MOODLE_19_STABLE
In order to ignore the new directory in your working area, add the line
/mod/certificate
to either
.gitignore
or to
.git/info/exclude
(I prefer the later)
$ cd ~/public_html/moodle19
$ echo /mod/certificate >> .git/info/exclude

Do not forget you have to manually include the certificate in all deployment processes as discussed above. As it is not part of your moodle19 checkout, it will not be pushed neither archived etc. If I understand it correctly, git submodules feature can be handy here but I haven't played with it yet.

Credits

A big thank you to Penny, Dan and Nigel for helping me with my first steps with git. Kudos for Martín and others from Catalyst for their contribution to the git-cvs integration tools.

See also