Note: You are currently viewing documentation for Moodle 1.9. Up-to-date documentation for the latest stable version is available here: Tracking Moodle CVS with git.

Obsolete:Tracking Moodle CVS with git: Difference between revisions

From MoodleDocs
(removed cogito commands - cogito is obsolete, and the git interface is quite friendly now)
m (removing categories as page is obsolete)
 
(44 intermediate revisions by 12 users not shown)
Line 1: Line 1:
Note: Work in progress. These are notes for users that are comfortable with CVS and other SCMs, doing branching, merging, etc.
==Introduction==
==Introduction==


Line 7: Line 5:
GIT was developed by Linus Torvalds specifically for the Linux Kernel team. It is fast, fast, fast. Its usage is slightly different from CVS/SVN, but, if kernel developers can handle it ''Moodle developers will find it '''easy''' '' ;-)
GIT was developed by Linus Torvalds specifically for the Linux Kernel team. It is fast, fast, fast. Its usage is slightly different from CVS/SVN, but, if kernel developers can handle it ''Moodle developers will find it '''easy''' '' ;-)


Git is packaged for most major operating systems and an up to date list of packages can be found on [http://git.or.cz/ git.or.gz].
Caveat: Git is not particularly easy to use or understand beyond trivial use. It was designed with a great deal of flexibility in mind and you will never get one answer to "how do I....". It is '''well worth''' spending some time to understand the basics of how git works. It will pay many times over. There are some very good online resources (see the end of this article).
 
Git is packaged for most major operating systems and an up to date list of packages can be found on [http://git.or.cz/ git.or.cz].


==Obtaining git==
==Obtaining git==


You'll want:
You'll want:
* [http://www.kernel.org/pub/software/scm/git/ Git] (and its deps). It includes '''gitk''', a very good UI for visualising project history.
* [http://git.or.cz/ Git] (and its deps). It includes '''gitk''', a very good UI for visualising project history.
* [http://www.cobite.com/cvsps/ cvsps] (a dependency of git-cvsimport make sure you have the latest release)
* an additional git porcelain (frontend)
** [http://sourceforge.net/projects/qgit qgit] a nice GUI to view the project history, make commits, etc. Recommended! Note: qgit is tricky to compile by hand, get a .deb or an .rpm
** [http://www.procode.org/stgit/ stgit] (StackedGIT is mainly for users doing heavy cherry picking. Only recommended for very advanced SCM users.)
 
Git is still developing quickly, but good mature versions are already available in packaged format. This guide is written with GIT v1.4 and Cogito v0.17 in mind. Ubuntu and Debian Backports have up-to-date packages for GIT, Cogito and related packages. RPMs are usually also available via YUM. If you want to be on the bleeding/leading edge you can follow the ''maint'' or ''master'' branches of the GIT development code.
 
==Importing CVS==


To begin we must import the CVS repository into a local git filesystem using [http://www.kernel.org/pub/software/scm/git/docs/git-cvsimport.html git-cvsimport].
You might also find the additional git porcelain (frontend) useful:
* [http://sourceforge.net/projects/qgit qgit] a nice GUI to view the project history, make commits, etc. Recommended! Note: qgit is tricky to compile by hand, get a .deb or an .rpm
* [http://www.procode.org/stgit/ stgit] (StackedGIT is mainly for users doing heavy cherry picking. Only recommended for very advanced SCM users.)


#!/bin/bash
Git is still developing very quickly, but good mature versions are already available in packaged format. This guide is written with GIT v1.5.3 in mind. Earlier versions work somewhat differently and you are unlikely to have a positive experience. You can check your version in the usual manner:
CVSROOT=:pserver:anonymous@uk.cvs.moodle.org:/cvsroot/moodle
MODULE=moodle
INSTALLDIR=moodle
git-cvsimport -p x -v -k -o cvshead -d $CVSROOT -C $INSTALLDIR $MODULE &> cvsimport.log


(When might one want to use the -m -u and -s flags?)
    git --version
-u purely cosmetic
-s slashes can cause problems but this doesn't seem to be the case with moodle
-m ?


This will run for a '''very''' long time as it checks out every revision of each file (including all branches).  However, if you run the command after a successful initial run it will simply get the new revisions since it was last run.  This creates a new git repository which can then be cloned and worked upon.
== Downloading Moodle CVS History for git ==


At the time of writing the size of a new Moodle git repository was approximately 866MB ''unpacked'', and 75MB packed. Newer versions of git pack the project during import, so your resulting repository plus checkout should be a bit over 100MB. Still, it is a good idea to pack the repository to make it smaller and faster, running
There are currently two methods of retrieving Moodle CVS history for git:
 
* Clone the git import found at [http://git.moodle.org/gw?p=moodle.git;a=summary git://git.moodle.org/moodle.git]
git-repack -a -d
* Run a local cvs import into git. ( Instructions for this can be found in [[Development:Importing Moodle CVS history into git]] )
 
With the -o cvshead flag, the "HEAD" of the CVS repo will appear as 'cvshead' in GIT, making things a lot clearer.
 
==Importing CVS Faster==
 
For initial imports, it is highly recommended that you fetch the CVS repository from one of Moodle [[CVS_for_Administrators#CVS_Servers | cvs servers]] and run the initial cvsimport against your local copy of the repository.
 
:: For the more adventurous: Keith Packard wrote another importer that does a better job at the initial import, called parsecvs. I use parsecvs for the initial import and then git-cvsimport daily for the incremental imports. --[[User:Martin Langhoff|Martin Langhoff]] 15:04, 19 July 2006 (WST)
 
:: It took me a few minutes to find the info in the Sourceforge docs on how to fetch the raw CVS repo via rsync. Just type 'rsync -av "rsync://moodle.cvs.sourceforge.net/cvsroot/moodle/moodle/*" .' (without the single quotes) and you'll be done --[[User:Iñaki Arenaza|Iñaki Arenaza]] 17:54, 25 December 2006 (CST).
 
:: This rsync command line doesn't work as-is anymore, since the repository was moved from sourceforge. Some mirrors (at least es.cvs.moodle.org at this time) provide rsync, but some don't (like eu.cvs.moodle.org). Find the best mirror for you and update the rsync call accordingly.
 
After the initial import, you can run git-cvsimport against your favorite cvs.moodle.org mirror repository, or update your local copy via rsync as sas demonstrated in [http://www.progsoc.org/~wildfire/git/update-repo.sh this script].


==Creating a Working Copy==
==Creating a Working Copy==


In order to use your repository you must clone yourself a working copy using [http://www.kernel.org/pub/software/scm/git/docs/git-clone.html git-clone].
In order to use your repository you must clone yourself a working copy using [http://www.kernel.org/pub/software/scm/git/docs/git-clone.html git clone].


(Let's assume your destination directory above was /pub/scm/moodle)
(Let's assume your destination directory is ~/src/moodle)


  mkdir ~/git/moodle
  git clone git://git.moodle.org/moodle.git ~/src/moodle
cd ~/git/moodle
git clone /pub/scm/moodle.git


This will clone the git filesystem you created from the CVS import into a working copy that you may make changes to and commit to. You can then get updates from the main git repository that will be merged in amongst the changes to your local copy.
This will clone the moodle git repostitory into a local working copy that you may make changes to and commit to. This initial clone will take some time, but note that once you have cloned this repository, you will have the complete moodle source code history locally and can easily browse and checkout any commit in moodle history without a network connection.


==Using git effectively with Moodle==
==Using git effectively with Moodle==
Line 73: Line 44:
If you have a local git copy of the Moodle CVS repository you will have a number of heads such as: '''MOODLE_15_STABLE''', '''MOODLE_16_STABLE''', and so on.  If you wish to create a new branch for your local customisations, you can base it off an existing HEAD (one of the CVS branches that exist in the main Moodle repository).  If you wish to create a new local branch, '''mymoodle''',  based off the current stable 1.6 code you would:
If you have a local git copy of the Moodle CVS repository you will have a number of heads such as: '''MOODLE_15_STABLE''', '''MOODLE_16_STABLE''', and so on.  If you wish to create a new branch for your local customisations, you can base it off an existing HEAD (one of the CVS branches that exist in the main Moodle repository).  If you wish to create a new local branch, '''mymoodle''',  based off the current stable 1.6 code you would:


  cd /pub/scm/moodle.git
  cd ~/src/moodle
  git branch mymoodle MOODLE_16_STABLE
  git branch mymoodle origin/MOODLE_16_STABLE
  git branch ''# lists all the current branches, should show '''mymoodle'''
  git branch ''# lists all the current branches, should show '''mymoodle'''
  git checkout mymoodle ''# you are now working on the '''mymoodle''' branch''
  git checkout mymoodle ''# you are now working on the '''mymoodle''' branch''
  ''# add some new files (i.e. themes, blocks, etc.)''
  ''# add some new files (i.e. themes, blocks, etc.)''
  cg-add -r .
  git add .
  cg-status ''# Will list your uncommited files''
  git status ''# Will list your uncommited files''
  cg-commit -m "Added base customisations."
  git commit -m "Added base customisations."


To return to the '''master''' HEAD, equivalent to the main Moodle CVS HEAD branch, you would have to use:
To return to the '''master''' HEAD, equivalent to the main Moodle CVS HEAD branch, you would have to use:
Line 86: Line 57:
  git branch master
  git branch master


If you were running a nightly script to pull new Moodle CVS updates into your main git repository via git-cvsimport, you should also merge any new changes into your local branch:
To get updated from cvs/catalyst


  cd /pub/scm/moodle.git
  cd ~/src/moodle
git fetch # fetches changes from upstream repostiroy
  git checkout mymoodle
  git checkout mymoodle
  cg-merge MOODLE_16_STABLE
  git merge origin/MOODLE_16_STABLE # merges changes from upstream


If you wanted to use this custom local branch to create a new client install, you would use:
If you wanted to use this custom local branch to create a new client install, you could use:


  cg-clone /pub/scm/moodle.git#mymoodle /home/someclient/public_html/
  git clone ~/src/moodle /home/someclient/public_html/
cd /home/someclient/public_html/
git branch mymoodle origin/mymoodle
git checkout mymoodle


This would create a new git repository from your branch that could be used to do a client install which can, in turn, have the client's custom files added to it.  This entire process can be automated so that your '''mymoodle''' branch is tracking upstream changes from the Moodle CVS ('''MOODLE_16_STABLE''' in this example) and your client installs are tracking changes against your local branch.
This would create a new git repository from your branch that could be used to do a client install which can, in turn, have the client's custom files added to it.  This entire process can be automated so that your '''mymoodle''' branch is tracking upstream changes from the Moodle CVS ('''MOODLE_16_STABLE''' in this example) and your client installs are tracking changes against your local branch.
Line 105: Line 80:


If the patches merge cleanly, future updates will recognise the already-applied patch. In that sense the merger in git is smarter than the merger in Cogito, consider using git-pull rather than cg-update. Note that if you see conflicts using git-pull you will have to work a bit harder to resolve them. Make sure you are familiar with handling merge conflicts with pure git before trying this.
If the patches merge cleanly, future updates will recognise the already-applied patch. In that sense the merger in git is smarter than the merger in Cogito, consider using git-pull rather than cg-update. Note that if you see conflicts using git-pull you will have to work a bit harder to resolve them. Make sure you are familiar with handling merge conflicts with pure git before trying this.
This is how I use git-cvsexportcommit: I have two directories - one with my Moodle git clone, the second with the CVS checkout. I commit a patch on a branch in git. Then I go into the CVS checkout directory and use something like
$ GIT_DIR=../moodle-19/.git git cvsexportcommit b38cb61
$ cvs commit -F .msg 'file1.php' 'path/to/file2.php'
Here "b38cb61" is the commit id I want to merge into CVS. See the cvsexportcommit output as it tells you what parameters to use during cvs commit.


==Preparing to merge large patch series or porting over to the next Moodle release==
==Preparing to merge large patch series or porting over to the next Moodle release==
Line 113: Line 95:


If you are using git-am, make sure you learn how to deal with merge conflicts, and the use of git-update-index.
If you are using git-am, make sure you learn how to deal with merge conflicts, and the use of git-update-index.
== IDE support for git ==
=== Eclipse ===
* [http://www.eclipse.org/egit/ JGit/EGit]
=== NetBeans ===
* [http://code.google.com/p/nbgit/ NBgit: git plugin for NetBeans] - maturity and git version are unknown,
* Native [http://netbeans.org/projects/versioncontrol/pages/Git_main Git support] is expected in NetBeans soon, they are [http://blogs.sun.com/netbeansphp/entry/netbeans_git_support_help_with collecting feedback] on [http://netbeans.org/projects/versioncontrol/pages/Git_main#Development_Documents specification] from developers, so anyone is welcome to contribute.
: See also [[Development:Setting up Netbeans#Git with NetBeans]].
=== TortoiseGit ===
* [http://code.google.com/p/tortoisegit/ TortoiseGit]
==Git Tips==
* See [[Development:Git tips]] for git tips to help you with moodle development.
* see [[Development:Tutorial_on_using_git_in_Moodle_development]] for yet another tutorial


==Helpful Documentation==
==Helpful Documentation==


* [http://wellington.pm.org/archive/200510/git/ cogito tutorial by martin langhoff]
* [http://www.kernel.org/pub/software/scm/git/docs/tutorial.html git tutorial]
* [http://www.kernel.org/pub/software/scm/git/docs/gitcvs-migration.html git for CVS users]
* [http://www.kernel.org/pub/software/scm/git/docs/gitcvs-migration.html git for CVS users]
* [http://www.kernel.org/pub/software/scm/git/docs/git-cvsimport.html git-cvsimport man page]
* [http://www.kernel.org/pub/software/scm/cogito/docs/ cogito man pages]
* [http://www.kernel.org/pub/software/scm/git/docs/ git man pages]
* [http://www.kernel.org/pub/software/scm/git/docs/ git man pages]
* [http://vger.kernel.org/vger-lists.html  The Git mailing list]
* [http://vger.kernel.org/vger-lists.html  The Git mailing list]
* [http://git.or.cz/gitwiki GitWiki]
* [http://git.or.cz/gitwiki GitWiki]
* [http://progit.org/book/ Pro Git Book] (excellent free online book)
* [http://www.kernel.org/git/ kernel.org Gitweb interface] (Browse the Git and associated tools' repositories)
* [http://www.kernel.org/git/ kernel.org Gitweb interface] (Browse the Git and associated tools' repositories)
* [http://wiki.sourcemage.org/Git_Guide Straightforward Git guide] (for Git >= 1.5)
* [http://wiki.sourcemage.org/Git_Guide Straightforward Git guide] (for Git >= 1.5)
* [http://www.newartisans.com/blog_files/git.from.bottom.up.php Git from the bottom up]
** [http://lwn.net/Articles/210045/ Branching and merging with git]


==Moodle Forum Discussions==
==Moodle Forum Discussions==
Line 134: Line 132:
* [http://moodle.org/mod/forum/discuss.php?d=66412 Moving to git, need serious help]
* [http://moodle.org/mod/forum/discuss.php?d=66412 Moving to git, need serious help]
* [http://moodle.org/mod/forum/discuss.php?d=99763 The continuing story of moving Moodle towards Git]
* [http://moodle.org/mod/forum/discuss.php?d=99763 The continuing story of moving Moodle towards Git]
[[Category:Developer|Tracking Moodle CVS with git]]

Latest revision as of 08:36, 23 December 2010

Introduction

If you plan on doing complicated or sustained development on Moodle, you will benefit from a local revision control system. Centralized systems like CVS and SVN have limited capabilities for tracking vendor branches. For the ultimate experience you will want to use a system with distributed capabilities like git. Alternative tools for this include Mercurial, Darcs and SVK.

GIT was developed by Linus Torvalds specifically for the Linux Kernel team. It is fast, fast, fast. Its usage is slightly different from CVS/SVN, but, if kernel developers can handle it Moodle developers will find it easy  ;-)

Caveat: Git is not particularly easy to use or understand beyond trivial use. It was designed with a great deal of flexibility in mind and you will never get one answer to "how do I....". It is well worth spending some time to understand the basics of how git works. It will pay many times over. There are some very good online resources (see the end of this article).

Git is packaged for most major operating systems and an up to date list of packages can be found on git.or.cz.

Obtaining git

You'll want:

  • Git (and its deps). It includes gitk, a very good UI for visualising project history.

You might also find the additional git porcelain (frontend) useful:

  • qgit a nice GUI to view the project history, make commits, etc. Recommended! Note: qgit is tricky to compile by hand, get a .deb or an .rpm
  • stgit (StackedGIT is mainly for users doing heavy cherry picking. Only recommended for very advanced SCM users.)

Git is still developing very quickly, but good mature versions are already available in packaged format. This guide is written with GIT v1.5.3 in mind. Earlier versions work somewhat differently and you are unlikely to have a positive experience. You can check your version in the usual manner:

   git --version

Downloading Moodle CVS History for git

There are currently two methods of retrieving Moodle CVS history for git:

Creating a Working Copy

In order to use your repository you must clone yourself a working copy using git clone.

(Let's assume your destination directory is ~/src/moodle)

git clone git://git.moodle.org/moodle.git ~/src/moodle

This will clone the moodle git repostitory into a local working copy that you may make changes to and commit to. This initial clone will take some time, but note that once you have cloned this repository, you will have the complete moodle source code history locally and can easily browse and checkout any commit in moodle history without a network connection.

Using git effectively with Moodle

Git can be used to intelligently track changes for creating client installs based off your own local, base, customisations which in turn tracks changes to the main Moodle CVS.

If you have a local git copy of the Moodle CVS repository you will have a number of heads such as: MOODLE_15_STABLE, MOODLE_16_STABLE, and so on. If you wish to create a new branch for your local customisations, you can base it off an existing HEAD (one of the CVS branches that exist in the main Moodle repository). If you wish to create a new local branch, mymoodle, based off the current stable 1.6 code you would:

cd ~/src/moodle
git branch mymoodle origin/MOODLE_16_STABLE
git branch # lists all the current branches, should show mymoodle
git checkout mymoodle # you are now working on the mymoodle branch
# add some new files (i.e. themes, blocks, etc.)
git add  .
git status # Will list your uncommited files
git commit -m "Added base customisations."

To return to the master HEAD, equivalent to the main Moodle CVS HEAD branch, you would have to use:

git branch master

To get updated from cvs/catalyst

cd ~/src/moodle
git fetch # fetches changes from upstream repostiroy
git checkout mymoodle
git merge origin/MOODLE_16_STABLE # merges changes from upstream

If you wanted to use this custom local branch to create a new client install, you could use:

git clone ~/src/moodle /home/someclient/public_html/
cd /home/someclient/public_html/
git branch mymoodle origin/mymoodle
git checkout mymoodle 

This would create a new git repository from your branch that could be used to do a client install which can, in turn, have the client's custom files added to it. This entire process can be automated so that your mymoodle branch is tracking upstream changes from the Moodle CVS (MOODLE_16_STABLE in this example) and your client installs are tracking changes against your local branch.

Merging a patch from GIT into CVS

If you have cvs access, you can use the git-cvsexportcommit command to merge or cherry pick individual patches into upstream.

git-cvsexportcommit was initially written to do merges of NZOSVLE bugfixes into Moodle. Several tools and patches to GIT and Cogito come from the NZOSVLE team. Martin Langhoff 14:55, 19 July 2006 (WST)

If the patches merge cleanly, future updates will recognise the already-applied patch. In that sense the merger in git is smarter than the merger in Cogito, consider using git-pull rather than cg-update. Note that if you see conflicts using git-pull you will have to work a bit harder to resolve them. Make sure you are familiar with handling merge conflicts with pure git before trying this.

This is how I use git-cvsexportcommit: I have two directories - one with my Moodle git clone, the second with the CVS checkout. I commit a patch on a branch in git. Then I go into the CVS checkout directory and use something like

$ GIT_DIR=../moodle-19/.git git cvsexportcommit b38cb61
$ cvs commit -F .msg 'file1.php' 'path/to/file2.php'

Here "b38cb61" is the commit id I want to merge into CVS. See the cvsexportcommit output as it tells you what parameters to use during cvs commit.

Preparing to merge large patch series or porting over to the next Moodle release

When you have a large set of patches to apply to CVS, or a large set of custom patches to apply on top of the next release of Moodle, you can use git-format-patch to see what are your unmerged patches. git-format-patch tries hard to spot already-merged patches and skip them.

If you are applying patches to CVS, you can then use git-cvsexportcommit or plain old patch -p1 filename. If you are applying them to a new git branch (as you would to merge them on top of a new Moodle release) then you want to be on that new branch, and use git-am -3 -k filename.

If you are using git-am, make sure you learn how to deal with merge conflicts, and the use of git-update-index.

IDE support for git

Eclipse

NetBeans

See also Development:Setting up Netbeans#Git with NetBeans.

TortoiseGit

Git Tips

Helpful Documentation

Moodle Forum Discussions