Note: You are currently viewing documentation for Moodle 1.9. Up-to-date documentation for the latest stable version is available here: Tracking Moodle CVS with git.

Obsolete:Tracking Moodle CVS with git: Difference between revisions

From MoodleDocs
(added Justin's approximation of repo size)
m (removing categories as page is obsolete)
 
(82 intermediate revisions by 20 users not shown)
Line 1: Line 1:
Note: Work in progress. The information in the document has thus far been compiled by an individual who knows nothing of git or tracking Moodle CVS. The information has not been verified by an authoritative source. Do not follow these instructions. But do work to make them better.
==Introduction==
 
If you plan on doing complicated or sustained development on Moodle, you will benefit from a local revision control system. Centralized systems like [http://www.nongnu.org/cvs/ CVS] and [http://subversion.tigris.org/ SVN] have limited capabilities for tracking vendor branches. For the ultimate experience you will want to use a system with distributed capabilities like [http://git.or.cz/ git]. Alternative tools for this include [http://www.selenic.com/mercurial/ Mercurial], [http://www.darcs.net/ Darcs] and [http://svk.elixus.org/ SVK].
 
GIT was developed by Linus Torvalds specifically for the Linux Kernel team. It is fast, fast, fast. Its usage is slightly different from CVS/SVN, but, if kernel developers can handle it ''Moodle developers will find it '''easy''' '' ;-)


==Introduction==
Caveat: Git is not particularly easy to use or understand beyond trivial use. It was designed with a great deal of flexibility in mind and you will never get one answer to "how do I....". It is '''well worth''' spending some time to understand the basics of how git works. It will pay many times over. There are some very good online resources (see the end of this article).


If you plan on doing complicated or sustained development on Moodle, you will benefit from a local revision control system. Centralized systems like [http://www.nongnu.org/cvs/ CVS] and [http://subversion.tigris.org/ SVN] have limitied capabilities for tracking vendor branches. For the ultimate experience you will want to use a system with distributed capabilities, like [http://svk.elixus.org/ SVK] or (as covered in this article) [http://git.or.cz/ git]. Note: While it is possible to use git on Win32 via cygwin, the performance will suffer.
Git is packaged for most major operating systems and an up to date list of packages can be found on [http://git.or.cz/ git.or.cz].


==Obtaining git==
==Obtaining git==


You'll want:
You'll want:
* [http://www.kernel.org/pub/software/scm/git/ Git] (and its deps)
* [http://git.or.cz/ Git] (and its deps). It includes '''gitk''', a very good UI for visualising project history.
* [http://www.cobite.com/cvsps/ cvsps] (a dependency of git-cvsimport)
 
* an additional git porcelain (frontend)
You might also find the additional git porcelain (frontend) useful:
** [http://www.kernel.org/pub/software/scm/cogito/ Cogito] (offers an interface more familiar to CVS/SVN users)
* [http://sourceforge.net/projects/qgit qgit] a nice GUI to view the project history, make commits, etc. Recommended! Note: qgit is tricky to compile by hand, get a .deb or an .rpm
** [http://www.procode.org/stgit/ stgit] (what's this do?)
* [http://www.procode.org/stgit/ stgit] (StackedGIT is mainly for users doing heavy cherry picking. Only recommended for very advanced SCM users.)
 
Git is still developing very quickly, but good mature versions are already available in packaged format. This guide is written with GIT v1.5.3 in mind. Earlier versions work somewhat differently and you are unlikely to have a positive experience. You can check your version in the usual manner:
 
    git --version
 
== Downloading Moodle CVS History for git ==
 
There are currently two methods of retrieving Moodle CVS history for git:
* Clone the git import found at [http://git.moodle.org/gw?p=moodle.git;a=summary git://git.moodle.org/moodle.git]
* Run a local cvs import into git. ( Instructions for this can be found in [[Development:Importing Moodle CVS history into git]] )
 
==Creating a Working Copy==
 
In order to use your repository you must clone yourself a working copy using [http://www.kernel.org/pub/software/scm/git/docs/git-clone.html git clone].
 
(Let's assume your destination directory is ~/src/moodle)
 
git clone git://git.moodle.org/moodle.git ~/src/moodle
 
This will clone the moodle git repostitory into a local working copy that you may make changes to and commit to. This initial clone will take some time, but note that once you have cloned this repository, you will have the complete moodle source code history locally and can easily browse and checkout any commit in moodle history without a network connection.
 
==Using git effectively with Moodle==
 
Git can be used to intelligently track changes for creating client installs based off your own local, base, customisations which in turn tracks changes to the main Moodle CVS.
 
If you have a local git copy of the Moodle CVS repository you will have a number of heads such as: '''MOODLE_15_STABLE''', '''MOODLE_16_STABLE''', and so on.  If you wish to create a new branch for your local customisations, you can base it off an existing HEAD (one of the CVS branches that exist in the main Moodle repository).  If you wish to create a new local branch, '''mymoodle''',  based off the current stable 1.6 code you would:
 
cd ~/src/moodle
git branch mymoodle origin/MOODLE_16_STABLE
git branch ''# lists all the current branches, should show '''mymoodle'''
git checkout mymoodle ''# you are now working on the '''mymoodle''' branch''
''# add some new files (i.e. themes, blocks, etc.)''
git add  .
git status ''# Will list your uncommited files''
git commit -m "Added base customisations."
 
To return to the '''master''' HEAD, equivalent to the main Moodle CVS HEAD branch, you would have to use:
 
git branch master
 
To get updated from cvs/catalyst


Git is very heavily still in development at this time and using Git to track the changes to the Git sourcecode is highly recommended. Debian has packages available but they are very outdated compared with the latest stable version of Git. If you are actually using Git as a tool that you depend on you will want to get the latest changes and fixes as soon as they are released.
cd ~/src/moodle
git fetch # fetches changes from upstream repostiroy
  git checkout mymoodle
  git merge origin/MOODLE_16_STABLE # merges changes from upstream


==Importing CVS==
If you wanted to use this custom local branch to create a new client install, you could use:


To begin we must import the CVS repository into a local git filesystem using [http://www.kernel.org/pub/software/scm/git/docs/git-cvsimport.html git-cvsimport].
git clone ~/src/moodle /home/someclient/public_html/
cd /home/someclient/public_html/
git branch mymoodle origin/mymoodle
git checkout mymoodle


#!/bin/bash
This would create a new git repository from your branch that could be used to do a client install which can, in turn, have the client's custom files added to itThis entire process can be automated so that your '''mymoodle''' branch is tracking upstream changes from the Moodle CVS ('''MOODLE_16_STABLE''' in this example) and your client installs are tracking changes against your local branch.
CVSROOT=:pserver:anonymous@cvs.sourceforge.net:/cvsroot/moodle
  MODULE=moodle
INSTALLDIR=moodle
git-cvsimport -p x -v -k -d $CVSROOT -C $INSTALLDIR $MODULE &> cvsimport.log


(When might one want to use the -m -u and -s flags?)
==Merging a patch from GIT into CVS==
-u purely cosmetic
-s slashes can cause problems but this doesn't seem to be the case with moodle
-m ?


This will run for a '''very''' long time as it checks out every revision of each file (including all branches).  However, if you run the command after a successful initial run it will simply get the new revisions since it was last run. This creates a new git repository which can then be cloned and worked upon. At the time of writing the size of a new Moodle git repository was approximately 866MB.
If you have cvs access, you can use the [http://www.kernel.org/pub/software/scm/git/docs/git-cvsexportcommit.html git-cvsexportcommit] command to merge or cherry pick individual patches into ''upstream''.  


TODO: investigate the utility of downloading a CVS repo tarball from SourceForge as demonstraited in [http://www.progsoc.org/~wildfire/git/update-repo.sh this script].
::git-cvsexportcommit was initially written to do merges of NZOSVLE bugfixes into Moodle. Several tools and patches to GIT and Cogito come from the NZOSVLE team. [[User:Martin Langhoff|Martin Langhoff]] 14:55, 19 July 2006 (WST)


==Creating a Working Copy==
If the patches merge cleanly, future updates will recognise the already-applied patch. In that sense the merger in git is smarter than the merger in Cogito, consider using git-pull rather than cg-update. Note that if you see conflicts using git-pull you will have to work a bit harder to resolve them. Make sure you are familiar with handling merge conflicts with pure git before trying this.
 
This is how I use git-cvsexportcommit: I have two directories - one with my Moodle git clone, the second with the CVS checkout. I commit a patch on a branch in git. Then I go into the CVS checkout directory and use something like
 
$ GIT_DIR=../moodle-19/.git git cvsexportcommit b38cb61
$ cvs commit -F .msg 'file1.php' 'path/to/file2.php'
 
Here "b38cb61" is the commit id I want to merge into CVS. See the cvsexportcommit output as it tells you what parameters to use during cvs commit.
 
==Preparing to merge large patch series or porting over to the next Moodle release==
 
When you have a large set of patches to apply to CVS, or a large set of custom patches to apply on top of the next release of Moodle, you can use [http://www.kernel.org/pub/software/scm/git/docs/git-format-patch.html git-format-patch] to see what are your ''unmerged'' patches. git-format-patch tries hard to spot already-merged patches and skip them.
 
If you are applying patches to CVS, you can then use git-cvsexportcommit or plain old patch -p1 filename. If you are applying them to a new git branch (as you would to merge them on top of a new Moodle release) then you want to be on that new branch, and use git-am -3 -k filename.


In order to use your repository you must clone yourself a working copy using [http://www.kernel.org/pub/software/scm/git/docs/git-clone-pack.html git-clone-pack].
If you are using git-am, make sure you learn how to deal with merge conflicts, and the use of git-update-index.


(Let's assume your destination directory above was /pub/scm/moodle)
== IDE support for git ==


mkdir ~/git/moodle
=== Eclipse ===
cd ~/git/moodle
* [http://www.eclipse.org/egit/ JGit/EGit]
git clone-pack /pub/scm/moodle.git


This will clone the git filesystem you created from the CVS import into a working copy that you may make changes to and commit to. You can then get updates from the main git repository that will be merged in amongst the changes to your local copy.
=== NetBeans ===
* [http://code.google.com/p/nbgit/ NBgit: git plugin for NetBeans] - maturity and git version are unknown,
* Native [http://netbeans.org/projects/versioncontrol/pages/Git_main Git support] is expected in NetBeans soon, they are [http://blogs.sun.com/netbeansphp/entry/netbeans_git_support_help_with collecting feedback] on [http://netbeans.org/projects/versioncontrol/pages/Git_main#Development_Documents specification] from developers, so anyone is welcome to contribute.
: See also [[Development:Setting up Netbeans#Git with NetBeans]].


If you are using the Cogito tools, you can use [http://www.kernel.org/pub/software/scm/cogito/docs/cg-clone.html cg-clone] to do this instead:
=== TortoiseGit ===
* [http://code.google.com/p/tortoisegit/ TortoiseGit]


cg-clone /pub/scm/moodle.git ~/git/moodle
==Git Tips==
* See [[Development:Git tips]] for git tips to help you with moodle development.
* see [[Development:Tutorial_on_using_git_in_Moodle_development]] for yet another tutorial


==Helpful Documentation==
==Helpful Documentation==


* [http://www.kernel.org/pub/software/scm/git/docs/tutorial.html git tutorial]
* [http://www.kernel.org/pub/software/scm/git/docs/gitcvs-migration.html git for CVS users]
* [http://www.kernel.org/pub/software/scm/git/docs/cvs-migration.html git for CVS users]
* [http://www.kernel.org/pub/software/scm/git/docs/git-cvsimport.html git-cvsimport man page]
* [http://www.kernel.org/pub/software/scm/cogito/docs/ cogito man pages]
* [http://www.kernel.org/pub/software/scm/git/docs/ git man pages]
* [http://www.kernel.org/pub/software/scm/git/docs/ git man pages]
* [http://vger.kernel.org/vger-lists.html  The Git mailing list]
* [http://vger.kernel.org/vger-lists.html  The Git mailing list]
* [http://www.kernel.org/git/ kernel.org Gitweb interface] (Browse the Git and assosciated tools' repositories)
* [http://git.or.cz/gitwiki GitWiki]
* [http://progit.org/book/ Pro Git Book] (excellent free online book)
* [http://www.kernel.org/git/ kernel.org Gitweb interface] (Browse the Git and associated tools' repositories)
* [http://wiki.sourcemage.org/Git_Guide Straightforward Git guide] (for Git >= 1.5)
* [http://www.newartisans.com/blog_files/git.from.bottom.up.php Git from the bottom up]
** [http://lwn.net/Articles/210045/ Branching and merging with git]


==Moodle Forum Discussions==
==Moodle Forum Discussions==


* [http://moodle.org/mod/forum/discuss.php?d=91998 Tracking Moodle CVS with git (take2)]
* [http://moodle.org/mod/forum/discuss.php?d=42275 Tracking Moodle CVS with git]
* [http://moodle.org/mod/forum/discuss.php?d=42275 Tracking Moodle CVS with git]
* [http://moodle.org/mod/forum/discuss.php?d=34472 Merging Custom Moodle Code With Stable Releases]
* [http://moodle.org/mod/forum/discuss.php?d=34472 Merging Custom Moodle Code With Stable Releases]
 
* [http://moodle.org/mod/forum/discuss.php?d=66412 Moving to git, need serious help]
[[Category:Developer]]
* [http://moodle.org/mod/forum/discuss.php?d=99763 The continuing story of moving Moodle towards Git]

Latest revision as of 08:36, 23 December 2010

Introduction

If you plan on doing complicated or sustained development on Moodle, you will benefit from a local revision control system. Centralized systems like CVS and SVN have limited capabilities for tracking vendor branches. For the ultimate experience you will want to use a system with distributed capabilities like git. Alternative tools for this include Mercurial, Darcs and SVK.

GIT was developed by Linus Torvalds specifically for the Linux Kernel team. It is fast, fast, fast. Its usage is slightly different from CVS/SVN, but, if kernel developers can handle it Moodle developers will find it easy  ;-)

Caveat: Git is not particularly easy to use or understand beyond trivial use. It was designed with a great deal of flexibility in mind and you will never get one answer to "how do I....". It is well worth spending some time to understand the basics of how git works. It will pay many times over. There are some very good online resources (see the end of this article).

Git is packaged for most major operating systems and an up to date list of packages can be found on git.or.cz.

Obtaining git

You'll want:

  • Git (and its deps). It includes gitk, a very good UI for visualising project history.

You might also find the additional git porcelain (frontend) useful:

  • qgit a nice GUI to view the project history, make commits, etc. Recommended! Note: qgit is tricky to compile by hand, get a .deb or an .rpm
  • stgit (StackedGIT is mainly for users doing heavy cherry picking. Only recommended for very advanced SCM users.)

Git is still developing very quickly, but good mature versions are already available in packaged format. This guide is written with GIT v1.5.3 in mind. Earlier versions work somewhat differently and you are unlikely to have a positive experience. You can check your version in the usual manner:

   git --version

Downloading Moodle CVS History for git

There are currently two methods of retrieving Moodle CVS history for git:

Creating a Working Copy

In order to use your repository you must clone yourself a working copy using git clone.

(Let's assume your destination directory is ~/src/moodle)

git clone git://git.moodle.org/moodle.git ~/src/moodle

This will clone the moodle git repostitory into a local working copy that you may make changes to and commit to. This initial clone will take some time, but note that once you have cloned this repository, you will have the complete moodle source code history locally and can easily browse and checkout any commit in moodle history without a network connection.

Using git effectively with Moodle

Git can be used to intelligently track changes for creating client installs based off your own local, base, customisations which in turn tracks changes to the main Moodle CVS.

If you have a local git copy of the Moodle CVS repository you will have a number of heads such as: MOODLE_15_STABLE, MOODLE_16_STABLE, and so on. If you wish to create a new branch for your local customisations, you can base it off an existing HEAD (one of the CVS branches that exist in the main Moodle repository). If you wish to create a new local branch, mymoodle, based off the current stable 1.6 code you would:

cd ~/src/moodle
git branch mymoodle origin/MOODLE_16_STABLE
git branch # lists all the current branches, should show mymoodle
git checkout mymoodle # you are now working on the mymoodle branch
# add some new files (i.e. themes, blocks, etc.)
git add  .
git status # Will list your uncommited files
git commit -m "Added base customisations."

To return to the master HEAD, equivalent to the main Moodle CVS HEAD branch, you would have to use:

git branch master

To get updated from cvs/catalyst

cd ~/src/moodle
git fetch # fetches changes from upstream repostiroy
git checkout mymoodle
git merge origin/MOODLE_16_STABLE # merges changes from upstream

If you wanted to use this custom local branch to create a new client install, you could use:

git clone ~/src/moodle /home/someclient/public_html/
cd /home/someclient/public_html/
git branch mymoodle origin/mymoodle
git checkout mymoodle 

This would create a new git repository from your branch that could be used to do a client install which can, in turn, have the client's custom files added to it. This entire process can be automated so that your mymoodle branch is tracking upstream changes from the Moodle CVS (MOODLE_16_STABLE in this example) and your client installs are tracking changes against your local branch.

Merging a patch from GIT into CVS

If you have cvs access, you can use the git-cvsexportcommit command to merge or cherry pick individual patches into upstream.

git-cvsexportcommit was initially written to do merges of NZOSVLE bugfixes into Moodle. Several tools and patches to GIT and Cogito come from the NZOSVLE team. Martin Langhoff 14:55, 19 July 2006 (WST)

If the patches merge cleanly, future updates will recognise the already-applied patch. In that sense the merger in git is smarter than the merger in Cogito, consider using git-pull rather than cg-update. Note that if you see conflicts using git-pull you will have to work a bit harder to resolve them. Make sure you are familiar with handling merge conflicts with pure git before trying this.

This is how I use git-cvsexportcommit: I have two directories - one with my Moodle git clone, the second with the CVS checkout. I commit a patch on a branch in git. Then I go into the CVS checkout directory and use something like

$ GIT_DIR=../moodle-19/.git git cvsexportcommit b38cb61
$ cvs commit -F .msg 'file1.php' 'path/to/file2.php'

Here "b38cb61" is the commit id I want to merge into CVS. See the cvsexportcommit output as it tells you what parameters to use during cvs commit.

Preparing to merge large patch series or porting over to the next Moodle release

When you have a large set of patches to apply to CVS, or a large set of custom patches to apply on top of the next release of Moodle, you can use git-format-patch to see what are your unmerged patches. git-format-patch tries hard to spot already-merged patches and skip them.

If you are applying patches to CVS, you can then use git-cvsexportcommit or plain old patch -p1 filename. If you are applying them to a new git branch (as you would to merge them on top of a new Moodle release) then you want to be on that new branch, and use git-am -3 -k filename.

If you are using git-am, make sure you learn how to deal with merge conflicts, and the use of git-update-index.

IDE support for git

Eclipse

NetBeans

See also Development:Setting up Netbeans#Git with NetBeans.

TortoiseGit

Git Tips

Helpful Documentation

Moodle Forum Discussions