Development:Git for developers

From MoodleDocs

This document is for helping you get started on Moodle development with Git. For further details of Git, see Category:Git.

General workflow

Moodle development workflow with Git

In short, the Moodle development with Git looks like this:

  • You as the contributor commit changes into your personal repository at your computer
  • You push the changes into your public repository and ask for their inclusion via so called PULL request
  • Moodle integrators pull the changes from your public repository and if they like them, they put them into Moodle integration repository
  • The integrated change is tested and finally pushed into Moodle production repository
  • You update your local repository with all changes from the production repository and the next cycle may start again

This workflow runs in roughly weekly cycles. The integration happens on Monday and the testing on Tuesday. On Wednesday, the production repository moodle.git is usually updated with changes from the last development week.

Most Moodle developers have their public repositories hosted at Github. Alternatively you may want to try Gitorious or the legendary repo.or.cz. In the examples in this guide we assume you'll set up your public repository at Github.

Installing Git on your computer

Install Git on your computer. Most Linux distributions have Git available as a package to install. If you are on Mac, git-osx-installer installs it in few click.

Immediately after the installation, set your name and contact e-mail. The name and e-mail will become part of your commits and they can't be changed later once your commits are accepted into the Moodle code. Therefore we ask contributors to use their real names written in capital letters, eg "John Smith" and not "john smith" or even "john5677".

   git config --global user.name "Your Name"
   git config --global user.email yourmail@domain.tld

Unless you are the repository maintainer, it is wise to set your Git to not push changes in file permissions:

   git config --global core.filemode false

Setting-up the public repository

1. Go to Github and create an account.

2. Go to the official Moodle Github repository and click on the Fork button. You now have your own Github Moodle repository.

3. Now you need to set up your SSH public key, so you can push to your Github Moodle repository from your local Moodle repository. On Mac you can go on this Github help page. If you are on another system, go to your Github administration page, to the section SSH Public Keys, and you should see a link to a help page. Done? Good! That was the most difficult part!

Setting-up the local repository at your computer

Create a local clone repository of your Github repository. In a terminal:

   git clone git@github.com:YOUR_GITHUB_USERNAME/moodle.git YOUR_LOCAL_MOODLE_FOLDER

This command does several jobs for you. It creates a new folder, initializes an empty Git repository in it, sets your Github repository as the remote repository called 'origin' and makes a local checkout of the branch 'master' from it. The important point to remember now is that your Github repository is aliased as 'origin' for your local clone.

Keeping your public repository up-to-date

Fetching changes from upstream and pushing them to github

Your fork at Github is not updated automatically. To keep it synced with the upstream Moodle repository, you have to fetch the recent changes from the official moodle.git and push them to your public repository. To avoid problems with this it is strongly recommended that you never modify the standard Moodle branches. Remember: never commit directly into master and MOODLE_xx_STABLE branches. In other words, always create topic branches to work on. In Gitspeak, the master branch and MOODLE_xx_STABLE branches should be always fast-forwardable.

To keep your public repository up-to-date, we will register remote repository git://git.moodle.org/moodle.git under 'upstream' alias. Then we create a script to be run regularly that fetches changes from the upstream repository and pushes them to your public repository. Note that this procedure will not affect your local working directory.

To register the upstream remote:

   git remote add upstream git://git.moodle.org/moodle.git

The following commands can be used to keep the standard Moodle branches at your Github repository synced with the upstream repository. You may wish to store them in a script so that you can run it every week after the upstream repository is updated.

   git fetch upstream
   for BRANCH in MOODLE_19_STABLE MOODLE_20_STABLE master; do
       git push origin refs/remotes/upstream/$BRANCH:$BRANCH
   done

How it works

The git-fetch command does not modify your current working dir (your checkout). It just downloads all recent changes from a remote repository and stores them into so called remote-tracking branches. The git-push command takes these remote-tracking branches from upstream and pushes them to Github under the same name. Understanding this fully requires a bit knowledge of Git internals - see gitrevisions(7) man page.

Note there is no need to switch the local branch during this. You can even execute this via cron at your machine. Just note that the upstream repository updates typically just once a week.

Preparing a patch

As said earlier at this page, you never work on standard Moodle branches directly. Every time you are going to edit something, switch to a local branch. Fork the local branch off the standard branch you it should be merged to. So if you are working on a patch for 1.9 or 2.0, fork the branch off MOODLE_19_STABLE or MOODLE_20_STABLE, respectively. Patches for the next major version should be based on the master branch.

   git checkout -b MDL-xxxxx-nasty-bug origin/master

Note that if you forget to specify the starting point, the branch is based on the currently checked-out branch. It may not be what you want. It is recommended to always specify the starting point.

To check the current branch, run

   git branch

The current branch is highlighted.

Now go and fix the issue with your favorite IDE. Check the status of the files, view the change to be committed and finally commit the change:

   vim filename.php
   git status
   git diff
   git commit -a

Note that this is safe as the commit is recorded just locally, nothing is sent to any server yet (as it would in CVS). To see history of the commits, use

   git log

Once your local branch contains the change (note that it may consists of several patches) and you are happy with it, publish the branch at your public repository:

   git push

Because we did not specify explicit remote repository, the 'origin' is used. Because we did not specify the branch to push, the Git will use the current branch and push it to the remote repository under the same name (by default, this is a subject of your configuration. See push.default config variable).

Now as your branch is published, you can ask Moodle core developers to review it and eventually integrate it into the standard Moodle repository.

Checking if a branch has already been merged

After some time contributing to Moodle you would have a lot of branches both in your local repository and in your public repository. To prune their list and delete those that were accepted by upstream, use the following

   git fetch upstream                                      (1)
   git branch --merged upstream/master                     (2)
   git branch --merged upstream/MOODLE_20_STABLE           (3)

The command (1) fetches the changes from your upstream repository at git.moodle.org (remember that git-fetch does not modify your working dir so it is safe to run it whenever). Command (2) and (3) print all branches that are merged into the upstream master branch and MOODLE_20_STABLE branch, respectively. To delete these local branches, use

   git branch -d MDL-xxxxx-accepted-branch

The similar approach can be used to check the branches published at your origin repository at github.com

   git fetch origin                                        (1)
   git fetch upstream
   git branch -r --merged upstream/master                  (2)
   git branch -r --merged upstream/MOODLE_20_STABLE        (3)

The command (1) makes sure that you have all your branches from github.com recorded as the remote tracking branch locally. Commands (2) and (3) work the same as in the previous example but they list remote tracking branches only (see -r param). To delete a branch at github.com, use

   git push origin :MDL-xxxx-branch-to-delete

This syntax may look weird to you. However it is pretty logical. The general syntax of the git-push command is

   git push <repository> <source ref>:<target ref>

so deleting a remote branch can be understood as pushing an "empty (null) reference" to it.

Peer-reviewing someone else's code

To review a branch that someone else pushed into their public repository, you do not need to register a new remote (unless you work with such repository frequently, of course). Let us imagine your friend Alice pushed a work-in-progress branch called 'wip-feature' into her Github repository and asked you to review it. You need to know the read-only address of the repository and the name of the branch.

   git fetch git://github.com/alice/moodle.git wip-feature

This will download all required data and will keep the pointer to the tip of the wip-feature branch in a local symbolic reference FETCH_HEAD. To see what's there on that branch, use

   git log -p FETCH_HEAD

To create a new local branch called 'alice-wip-feature' containing the work by Alice, use

   git checkout -b alice-wip-feature FETCH_HEAD

To merge Alice's work into your current branch:

   git merge FETCH_HEAD

To see what would be merged into the current branch without actually modifying anything:

   git diff ...FETCH_HEAD

Rebasing a branch

Rebasing is a process when you cut off the branch from its current start point and transplant it to another point. Let us assume the following history exists:

         A---B---C topic
        /
   D---E---F---G master

From this point, the result of the command:

   git rebase master topic

would be:

                 A'--B'--C' topic
                /
   D---E---F---G master

and would end with 'topic' being your current branch.

You may be asked to rebase your branch submitted for the integration if the submitted branch was based on an outdated commit. The typical case is if you create a new branch as a fork off the upstream master branch on Tuesday. Then on Wednesday, the upstream master branch grows as all changes from the last integration cycle are merged to it. To make diff easy on Github for next weekly pull request review, you want to rebase your branch against the updated master.

   git rebase master MDL-xxxxx-topic-branch

Note that rebasing effectively rewrites the history of the branch. Do not rebase the branch if there is a chance that somebody has already forked it and based their own branch on it. For this reason, many Git tutorials discourage from rebasing any branch that has been published. However in Moodle, all branches submitted for integration are potential subject of rebase (even though we try to not to do it often) and you should not base your own branches on them.

Conflicts during rebase

During the rebase procedure, conflicts may appear. git-status commands reports the conflicted files. Explore them carefully and fix them in your editor (like you would do with CVS). Then add the files with 'git add' command and continue.

   vim conflicted.php
   git add conflicted.php
   git rebase --continue

Cherry-picking changes from one branch to another

Most bugs are fixed at a stable branch (like MOODLE_20_STABLE) and the fix must be prepared for the main development branch (master), too. In Moodle, we do not merge stable branches into the master one. So usually the contributor prepares two branches - one with the fix for the stable branch and the second with the fix for the master branch.

If you have a patch prepared on a local branch, it is possible to use git-cherry-pick command to re-apply the commits to another branch. Suppose the following scenario.

Let us have two local Git repositories ~/public_html/moodle20 containing local installation of Moodle 2.0 and ~/public_html/moodle21 with the local installation of Moodle 2.1dev. They both use your public repository at github.com as the origin. You have a branch in moodle20 called MDL-xxxx-topic_20_STABLE that contains several commits. Now you want to re-apply these commits to a branch MDL-xxxx-topic_master in moodle21.

   cd ~/public_html/moodle20
   git log origin/MOODLE_20_STABLE..MDL-xxxx-topic_20_STABLE       (1)
   cd ~/public_html/moodle21
   git checkout -b MDL-xxxx-topic_master origin/master             (2)
   git fetch  ~/public_html/moodle20 MDL-xxxx-topic_20_STABLE      (3)
   git cherry-pick origin/MOODLE_20_STABLE..FETCH_HEAD             (4)

The command (1) shows all commits that are added to the MDL-xxxx-topic_20_STABLE. These are the commits you want to re-apply to the another branch. The command (2) creates a new local branch forked off the current master. The command (3) fetches all data needed to re-apply commits from the topic branch and stores the pointer to the tip of that branch to FETCH_HEAD symbolic reference. The command (4) picks all commits between the MOODLE_20_STABLE branch and the FETCH_HEAD (that is the commits listed in (1)) and tries to apply them on the current branch.

See also