Note:

If you want to create a new page for developers, you should create it on the Moodle Developer Resource site.

Git Migration Workflows

From MoodleDocs
Warning: This page is no longer in use. The information contained on the page should NOT be seen as relevant or reliable.
Decide development workflow with git
Project state Discussing proposal
Tracker issue MDLSITE-858
Discussion TODO
Assignee Not assigned yet


Quite a number of developers are now familiar with git and are enthusiastic about the power and flexibility it brings to the development process. However, it was not decided yet which model exactly is going to be used for Moodle development and how are we going to migrate from CVS to git.

Git doesn't place constraints to one model of development which is both a feature and a hindrance for people understanding how the development process takes place. A number of workflows need to be developed in order that developers can understand how git would fit into Moodle development.

This page summarizes alternative workflows and describes the suggested one in more details.

Considered alternatives

Push model (CVS like)

In this model, developers push their commits to a central repository as happens with CVS now. Commit access is granted via SSH to a number of trusted users. Git commit hooks would be used to granular access control to various areas of the source code tree. Other external contributors can eventually publish their git repository branches or patches to developers with commit access for inclusion to the mainline.

This would not require many changes in the process itself as the workflow is similar to what we use at the moment. Switching to git would still offer many benefits like two level committing (locally commit first, then pushing to the repository that in turn allows interactive rebase to merge local commits etc). Also, developers could use some web interface (like github.com) to publish and discuss proposed changes before they are committed into the repository.

Committing everything directly into the branches in central repository leads to many problems: Bad commits cannot be undone, so they must be reverted one by one, which creates confusing histories and further error potential when you forget to revert part of a group of changes. Still, this open push model seems to be a good choice for plugins in contrib repository. However, some core developers would prefer more review and approval based process before changes are incorporated into Moodle core.

See A Git Workflow for Agile Teams for more detailed workflow description.

Pull model with integrator role

In this model, just a small group of developers have write access to the official Moodle git repository. In fact, the single integrator would be enough but for practical reasons (holiday, being sick or aunt Ermer visiting), three of four would be good. These gatekeepers integrate pull requests from other subsystem maintainers and patches from developers. This model is used for Linux kernel, for example. Draft of such a workflow specification follows.

Moodle core development workflow proposal

This section describes a suggested model of Moodle core development as it could look like when we migrate to git. It was inspired by [1] and you are encouraged to read that page, too.

Repositories

The official Moodle repository is git.moodle.org/moodle.git. Only limited group of developers have write access to it - let us call them Martin, Eloy and Petr, for example. There is a clone of this repository at github.com where every commit is automatically pushed into via commit hooks.

All other developers (yes, including those working for HQ, too) and contributors have their own clones of moodle.git. They can be published at git.moodle.org (eg git.moodle.org/moodle-mudrd8mz.git) or github.com (eg github.com/mudrd8mz/moodle.git) or both or actually anywhere else from where the integrators can fetch the changes from.

Integration branches

The official moodle.git repository contains the following integration branches:

  • MOODLE_18_STABLE, MOODLE_19_STABLE, MOODLE_20_STABLE etc. track the commits that should go into the next 'maintenance release', i.e., update of the last released stable version (like 1.9.9 or 2.0.1). In current CVS, we already have these branches.
  • master tracks the commits that should go into the next major release. In current CVS, this is known as HEAD.

Eventually, we might introduce new branch next intended as unstable testing branch for topics being tested for stability before they get into master. Such branch is being used in projects using git as their master is pretty stable enough to let the product actually run and pass tests.

Graduation, merging and cherry-picking

Every branch is usually a direct descendant of the one above it. Conceptually, the feature enters at an unstable branch (usually next), and graduates to master for the next release once it is considered stable enough.

Fixes are committed into the oldest supported branch that require them. Then (periodically) the integration branches are merged upwards into each other.

If there is fix applied to e.g. master that is also required in MOODLE_xx_STABLE maintenance branch, it will be cherry-picked (using git cherry-pick) downwards. This will happen a few times and is nothing to worry about unless we do it very frequently.

Other developers (non-integrators)

Nobody but integrators have write access into the official moodle.git repository so the work done by other developers must be pulled from their clones. Developers work on their local repositories and publish changes in their public trees (so there is David's Moodle tree, Petr's tree, Sam's tree, Jerome's tree etc.).

Any nontrivial feature usually requires several patches to implement, and may get extra bugfixes or improvements during its lifetime. Developers make a side branch for every topic (feature or bugfix). They fork it off at the oldest integration branch that they will eventually want to merge it into.

Example:

$ cd ~/public_html/moodle20/                       // go to your clone
$ git checkout MOODLE_20_STABLE                    // switch the checkout into the local stable branch
$ git fetch                                        // fetch recent changes from origin repo
$ git merge --ff-only origin/MOODLE_20_STABLE      // merge the origin stable branch into your local branch
                                                   // developer never works on MOODLE_20_STABLE directly so the
                                                   // fast-forward does not fail
$ git checkout -b MDL-29123-file_api_regression    // fork off a topic branch
$ vim foo.php                                      // work on the topic
$ git commit -a                                    // commit often
$ vim bar.php                                      // and again - commit often
$ ...
$ git fetch                                        // fetch recent updates from upstream again
$ git rebase origin/MOODLE_20_STABLE               // as we have not published our work yet, we can rebase
$ git rebase -i origin/MOODLE_20_STABLE            // make sure the commits have nice commit messages, meld them if needed
$ git cherry origin/MOODLE_20_STABLE               // review that this is what we want to be merged
$ git push mymoodletree MDL-29123-file_api_regression:forupstream/MDL-29123-file_api_regression
                                                   // publish the polished branch at own repository somewhere
                                                   // here the name prefix is used to make it clear the branch
                                                   // is ready to be merged

At this moment, the developer has to ask the integrators to merge their branch. It must be clear that the pull request is for MOODLE_20_STABLE integration branch. The developer can use git request-pull command or github.com pull request feature or direct email or JIRA comment or any other agreed method.

From: David Mudrak
To: pull-request@moodle.org
Subject: [PULL REQUEST] MDL-29123 for Moodle 2.0.x

Please pull this bugfix into MOODLE_20_STABLE from
   git://git.mudrak.name/moodle-mudrd8mz.git forupstream/MDL-29123-file_api_regression

Then the integrator just fetches from the repository and tries to merge into their local MOODLE_20_STABLE.

$ git checkout MOODLE_20_STABLE
$ git pull git://git.mudrak.name/moodle-mudrd8mz.git forupstream/MDL-29123-file_api_regression

If the branch does not merge cleanly, they can either fix it on their own (in case of trivial conflicts) or ask the author to fix the conflict (not by rebasing but by merging MOODLE_20_STABLE into the topic branch. If the branch merges cleanly and the integrator likes it, they will push their local MOODLE_20_STABLE into the main moodle.git repo and merge the branch into master and next.

If the topic has evolved further in the meantime, it is merged again. Note that you do not necessarily have to merge it to the oldest integration branch first. For example, you can first merge a bugfix to next, give it some testing time, and merge to a maintenance branch when you know it works well.

If the developer finds she needs new features from the branch other to continue working on your topic, she merges other to topic.

Topic branch that has been merged elsewhere must not be rebased any more. See the section on RECOVERING FROM UPSTREAM REBASE in git-rebase.

We should point out that regular merging an integration branch for no real reason into your topics — and by extension, merging anything upstream into anything downstream on a regular basis — should be done at well-defined points only (like some tagged points). Do not merge to downstream except with a good reason: upstream API changes affect your branch; your branch no longer merges to upstream cleanly; etc. Otherwise, the topic that was merged to suddenly contains more than a single (well-separated) change. The many resulting small merges will greatly clutter up history. Anyone who later investigates the history of a file will have to find out whether that merge affected the topic in development. An upstream might even inadvertently be merged into a "more stable" branch. And so on.

Notes

  • We may find using the next branch as over-processing step. Instead, the master could be used as the entry point for new features.
  • It may look as the integration is hard and time consuming job. In fact, if there are subsystem maintainers known, the procedure is quite routine. For example, when David asks for the inclusion of some commits that change files in mod/workshop only, just a basic security review (like missing require_login()) or wrong coding style can be quickly done.

Contrib plugins

Every plugin in the current CONTRIB repository is converted into separated git repository. If the maintainer is known and she is willing to use pull model as well, the procedure is very similar to the core one. If there is no maintainer known, trusted developers can be granted with write access to the plugin repository and the procedure follows the push model.

Every plugin should contrain README.txt or CONTRIBUTION.txt or similar file with clear instructions on how to contribute to the plugin.

CVS read-only mirrors

Snapshots of moodle.git branches are produced daily, hourly or even after every commit and we maintain these snapshots in a CVS repository. That way we offer a solution for admins who use CVS for getting Moodle sources and do not want to switch to git for some reason.