Difference between revisions of "Process"

Jump to: navigation, search
(Adding clause for component leads)
(Adding clause for component leads)
Line 23: Line 23:
 
===Component leads===
 
===Component leads===
  
[http://tracker.moodle.org/browse/MDL#selectedTab=com.atlassian.jira.plugin.system.project%3Acomponents-panel Component leads] are developers with some responsibility for particular components (plugins or modules) in Moodle.  They have the authority to decide that a particular fix is suitable and complete enough to be considered for integration in Moodle core, by adding their git branches to the integration queue.  This can happen at any time.
+
[http://tracker.moodle.org/browse/MDL#selectedTab=com.atlassian.jira.plugin.system.project%3Acomponents-panel Component leads] are developers with some responsibility for particular components (plugins or modules) in Moodle.  They have authority to decide that a particular fix is suitable and complete enough to be considered for integration in Moodle core and should be called upon to complete peer reviews for code in their components.
  
 
===Integrators===
 
===Integrators===
Line 131: Line 131:
 
You need to fill in the information about your public git repository and which branches the fixes are on.
 
You need to fill in the information about your public git repository and which branches the fixes are on.
  
This would be a good time to fill in the testing instructions for how to verify your fix is correct. You may also wish to add a comment in the bug. (... or should you put that information in the commit comment?)
+
This would be a good time to fill in the testing instructions for how to verify your fix is correct. You may also wish to add a comment in the bug.
 +
 
 +
Component leads should put issues, which affect code in their components, up for peer review to allow interested parties to provide feedback. However, if it is not reviewed in a week, a component lead may send the issue to integration. All other developers, including people who are component leads but working outside their component, should have their issues peer reviewed before they are sent to integration.
  
===Component lead does the peer review===
+
===Peer review===
  
The [http://tracker.moodle.org/browse/MDL#selectedTab=com.atlassian.jira.plugin.system.project%3Acomponents-panel component lead] should peer-review the change. They will either give you comments on the code and if it needs more work.
+
The [http://tracker.moodle.org/browse/MDL#selectedTab=com.atlassian.jira.plugin.system.project%3Acomponents-panel component lead] should peer-review the change. If there is no component lead for an affected component, any other recognised developer may complete the peer review. The peer reviewer will either give you comments on the code and if it needs more work.
  
 
Ideally, peer reviews should check all items in the [[Peer reviewing checklist]].
 
Ideally, peer reviews should check all items in the [[Peer reviewing checklist]].
Line 170: Line 172:
  
 
This would be a good time to fill in the testing instructions for how to verify your fix is correct. You may also wish to add a comment in the bug. (... or should you put that information in the commit comment?)
 
This would be a good time to fill in the testing instructions for how to verify your fix is correct. You may also wish to add a comment in the bug. (... or should you put that information in the commit comment?)
 
Component leads should put issues affecting code in their components up for peer review to allow interested parties to provide feedback. However, if it is not reviewed in a week, a component lead may send the issue to integration. All other developers, including people who are component leads but working outside their component, should have their issues peer reviewed before they are sent to integration.
 
  
 
===Component lead does the peer review===
 
===Component lead does the peer review===

Revision as of 07:12, 25 July 2013

This document summarises the various development processes used in developing Moodle. There are four main processes that overlap.

Integration workflow in the tracker

The Moodle tracker keeps track of the status of all bug fixes and new features.

We use a workflow that ensures that new code receives multiple reviews by different people before it is included into the core Moodle code.

Workflow.jpg

A number of roles make this work:

Users

Users report bugs and make feature requests directly in the tracker, by creating new issues with a summary and a description.

Developers

Developers work on the issues in the tracker to specify solutions and write code that implements these solutions. They will often ask other developers to "peer review" their code in the early stages to avoid problems later on.

While many of the developers work for Moodle.com, a large number are part of the global development community around Moodle. If you're interested in becoming a recognised developer, see Tracker groups and permissions.

Component leads

Component leads are developers with some responsibility for particular components (plugins or modules) in Moodle. They have authority to decide that a particular fix is suitable and complete enough to be considered for integration in Moodle core and should be called upon to complete peer reviews for code in their components.

Integrators

On Monday and Tuesday of each week, the integration team (a small team of senior developers employed by Moodle HQ) conducts a code-level review of all issues in the integration queue. This is often called the "pull" process. If the fix is judged appropriate they will integrate the code into our git integration repository for further testing and it gets added to the testing queue.

If they find problems they reject the issue and send it back to the developer for further work.

Testers

On Wednesday each week, testers look at all the issues in the testing queue, trying each fix and feature to make sure that it does actually fix the problem it was supposed to, and that there are no regressions.

If they find problems they reject the issue and integrators may remove it from the integration repository and push it back to the developer for further work.

Production maintainers

On Thursday each week, production maintainers merge all the issues that passed testing into the git production repository, and it becomes available for use on production systems via git, cvs, and download packages.

Stable maintenance cycles

Moodle releases regular updates of the stable version of the software to fix bugs and other issues. Releases like 2.2.1, 2.2.2, 2.2.3 etc only include fixes based on the latest major release (2.2) and never any significant new features or database changes.

At Moodle HQ there are teams of developers using the Scrum framework to work on these issues (as well as new features for major releases).

Minor release (point release) timing

After major releases there will be minor releases.

  • x.x.1 will occur approximately two months after each major release (eg. 2.x).
  • There will then be another point release every two months after that.

See the General release calendar for details.

Issue triage

Issue triage involves evaluating new issues, making sure that they are recorded correctly. One of the most important jobs triagers do is to identify issues that should be fixed in the stable branch. These are set with a priority ranging from "Trivial" up to "Blocker" and other features are checked.

At Moodle HQ there are currently teams working on FRONTEND issues (mostly related to user interface) and BACKEND issues (mostly related to APIs) and issues may be put on specific backlogs for these teams, as appropriate.

Scrum

At Moodle HQ, every three weeks, the FRONTEND and BACKEND teams take a number of the most urgent issues from their relevant backlogs to work on during a period known as a sprint.

At the start of a sprint there is a period of planning and estimation. All issues on the FRONTEND and BACKEND backlogs are given a relative rank that is based on issue features including priority, security, Partner interest, patches and votes. Issues are given a relative size in Story Points and these points are summed to allow the teams to determine how many issues they can work on in the sprint.

During the sprint, the team meets daily to discuss solutions and progress, as well as to organise testing and peer reviews of code. The team has a Scrum master to help everyone stay organised, to "unblock" any barriers to progress and to protect the team from distracting influences (mostly people attempting to add to the developers' workloads) during the sprint. The teams' work is documented publicly in the tracker.

Whenever a solution for an issue is finished, it is submitted into to the standard integration workflow process described above.

Major release cycles

Since Moodle 2.0, we have a policy of release major versions (eg 2.1, 2.2) every six months in May and November. See the General release calendar for more details.

Each release can be different, but generally the cycles work as follows.

Define roadmap

The product owner (Martin Dougiamas) defines the likely roadmap based on community wishes, third-party developments and important issues within the existing code.

Sometimes new features might be based on earlier features, sometimes they may be something developed by a third party that needs to be evaluated and sometimes it might be something completely new.

Planning and development

The FRONTEND and BACKEND teams, employed at Moodle HQ, work on specifications of major new features for approximately three to four weeks following a major release.

The process of new feature development is described below. When specifications are in place, new code is developed during sprints and goes through the standard weekly integration workflow described above.

Testing

During development, as new code is integrated, automated testing conducted at the code and interface levels, to make sure there are no regressions caused by new features.

In the last month before the release, a feature freeze is called (no new features can be added) and our QA team performs manual functional testing of Moodle features. The current list of functional tests is listed in the tracker in the MDLQA project. This list of tests is extended as new features are added, but it is also being reduced as more automated acceptance tests are developed.

For more details, see Testing.

Sprints

At Moodle HQ, development takes place in sprints. The sprints are three-week periods during which developers to focus on a fixed list of issues. Sprints are arranged within each release cycle as shown in the diagram below.

Dev sprint calendar.png

New feature development

Major new features in Moodle usually should go through the following process.

Specification

Developers should create a detailed spec (here in the developer docs) outlining their goals for the development and their design for meeting those goals. The more detail the better.

Developers should also create an issue in the tracker (linking to your docs) to keep track of the project status.

Community consultation

Get the community involved in looking at the spec to see if it meets their needs and to get further feedback. Please post in the Future major features forum on moodle.org. You could also blog/tweet about it etc.

Community developers proposing a new feature will want to talk with HQ core developers to make sure the ideas make sense, and possibly get some review on database design, architecture and so on.

Develop the code using Git

Develop your code on an open Git repository, like github.com. That enables people to see your code and to help you as it develops. Testers and early adopters also have the opportunity to try it early in the process and give you more valuable feedback.

It is essential that your code follows the Moodle Coding Guide.

Submit your code for peer review

Use the workflow buttons in the tracker.

You need to fill in the information about your public git repository and which branches the fixes are on.

This would be a good time to fill in the testing instructions for how to verify your fix is correct. You may also wish to add a comment in the bug.

Component leads should put issues, which affect code in their components, up for peer review to allow interested parties to provide feedback. However, if it is not reviewed in a week, a component lead may send the issue to integration. All other developers, including people who are component leads but working outside their component, should have their issues peer reviewed before they are sent to integration.

Peer review

The component lead should peer-review the change. If there is no component lead for an affected component, any other recognised developer may complete the peer review. The peer reviewer will either give you comments on the code and if it needs more work.

Ideally, peer reviews should check all items in the Peer reviewing checklist.

Submit the code for integration

This is much the same as for any Moodle code. See the information about the integration workflow above.

Fixing a bug

Bug fixes, and minor features or enhancements should go through the following process.

Make sure there is a tracker issue

Every change must have an issue in the tracker. If you are fixing a bug, there is probably one there already, but if not, create one.

Decide which branches the fix is required on

Bugs should normally be fixed on all the supported stable branches that are affected. New features should just go into master, but sometimes minor enhancements are made on the most recent stable branch.

Develop your change using git

Develop your fix and push the change to an open git repository, for example on github.com.

It is essential that your code follows the Moodle Coding Guide.

You will need to push one commit for each branch the fix needs to be applied to. Often people use branch names like wip_MDL-12345_2.1_brief_name so it is clear what each branch is. git cherry-pick can help with replicating the fix onto different branches.

Submit your code for peer review

Use the workflow buttons in the tracker.

You need to fill in the information about your public git repository and which branches the fixes are on.

This would be a good time to fill in the testing instructions for how to verify your fix is correct. You may also wish to add a comment in the bug. (... or should you put that information in the commit comment?)

Component lead does the peer review

The component lead should peer-review the change. They will either give you comments about why the code needs more work, or they will ...

Ideally, peer reviews should check all items in the Peer reviewing checklist.

Submit your code for integration

It will then be reviewed the following week by one of the integration team and either integrated or rejected. Once integrated, the fix will be tested, and then included in the next weekly release.

Security issues

Issues identified as security issues are resolved in a slightly different way, in order to achieve responsible disclosure.

  • Security issues should be labelled as "Minor" or "Serious" in order control visibility of the issue.
    • An issue reported with a security level of "Could be a security issue" should be evaluated as soon as possible and either set as "Minor" or "Serious" or the security level should be set to "None".
  • Solutions to security issues should not made available in public repositories.
  • The solution will not be integrated until the week before a minor release following the normal Release process. In short, the issue will be incorporated into the integration version, rebased, tested and made ready for release as a normal issue would, but not until as late as possible.
  • Details of security issues will be shared with registered admins with the minor release.
  • Details of security issues will not be publicly announced until one week after a minor release to allow admins to update.

How to remove a branch from Github

To remove a branch from Github, you can use the following command.

git push github :remote_branch

Where remote_branch is the name of your remote branch, for example 'wip-mdl-1234'. This effectively replaces the remote branch with nothing, removing the remote branch, but leaving the branch intact in your local Git repository. Please note that its likely that your commit will still exist on github due to the nature of git, so its best to avoid doing this in the first place.

See also