Note: This documentation is for Moodle 2.7. For up-to-date documentation see Process.

Development:Process: Difference between revisions

From MoodleDocs
mNo edit summary
 
(6 intermediate revisions by the same user not shown)
Line 1: Line 1:
We are planning a new development process for future core work on Moodle to integrate reviews at many levels throughout the process of code development.
This document summarises the various development processes used in developing Moodle. There are four main processes that overlap.
 
==Terms==


Here are some terms we'll use in this document:
Here are some terms we'll use in this document:
* STABLE = the next stable minor version of Moodle (eg 2.0.1, 2.0.2 ... etc) Note that database changes and new features are not allowed in STABLE
* STABLE = the next stable minor version of Moodle (eg 2.0.1, 2.0.2 ... etc)
* STABLE backlog = the queue of bugs to be fixed in STABLE  (eg 2.0.x)
* STABLE backlog = the queue of bugs to be fixed in STABLE  (eg 2.0.x)
* DEV = the next major version of Moodle (eg 2.1, 2.2, 2.3 ... etc)
* DEV = the next major version of Moodle (eg 2.1, 2.2, 2.3 ... etc)
* DEV backlog = everything to be looked at for upcoming major versions (eg 2.X)
* DEV backlog = everything to be looked at for upcoming major versions (eg 2.X)


==Overview of the process==


[[Image:Devprocess.jpg]]


==Integration workflow in the tracker==
The Moodle tracker keeps track of the status of all bug fixes and new features.
We use a workflow that ensures that new code receives multiple reviews by different people before it is included into the core Moodle code.
[[Image:Workflow.jpg]]


==Roles==
A number of roles make this work:


Let's look at the roles in the new system.  Note that one person may perform more than one of these roles, depending on their job description.  However, no-one should ever be reviewing their own code.
===Users===


Users report bugs and make feature requests directly in the tracker, by creating new issues with a summary and a description.


===User role===
===Developers===


# Uses Moodle
Developers work on the issues in the tracker to specify solutions and write code that implements these solutions.  They will often ask other developers to "peer review" their code in the early stages to avoid problems later on.
# Finds issues (both "bugs" and "suggestions")
# Report issues to tracker


While many of the developers work for Moodle.com, a large number are part of the global development community around Moodle.


===[[Development:Bug triage|Triage role]]===
===Component leads===


Initial staff: Eloy, Helen
Component leads are developers with some responsibility for particular components (plugins or modules) in Moodle.  They have the authority to decide that a particular fix is suitable and complete enough to be considered for integration in Moodle core, by adding their git branches to the integration queue.  This can happen at any time.


# Reads new issues coming into Moodle Tracker
===Integrators===
# Performs housekeeping, fixes spelling, removes duplicates and spam, updates priorities/assignees/components/descriptions/statuses.
# For items in core:
## If the issue is a bug in the current stable version (eg 2.0), assign "Fix version" as STABLE backlog
## If the issue is related to backup and restore, assign "Fix version" to STABLE backlog with high priority
## If the issue is a bug in the current stable version requiring database changes, assign "Fix version" to DEV backlog
## If the issue is a feature request, assign "Fix version" to DEV backlog


===Product Owner role===
On Monday of each week, the integration team (a small team of senior developers employed by Moodle HQ) conducts a code-level review of all issues in the integration queue.  This is often called the "pull" process.  If the fix is judged appropriate they will integrate the code into our git integration repository for further testing and it gets added to the testing queue.


Initial staff: Martin
If they find problems they reject the issue and send it back to the developer for further work.


# Monitors DEVBACKLOG, updates priorities to produce a queue
===Testers===
# Manages community discussions/feedback
# Maintains overall roadmap and goals based on issues + community
# Develops detailed specifications


===Scrum Master role===
On Tuesday each week, testers look at all the issues in the testing queue, trying each fix and feature to make sure that it does actually fix the problem it was supposed to, and that there are no regressions.


Initial staff: ???
If they find problems they reject the issue and integrators may remove it from the integration repository and push it back to the developer for further work.


# Coaches and helps the team to finish each sprints
===Committers===
# Holds sprint restrospective meetings
# Keeps documentation updated


===Moodle HQ Stable Maintenance team===
On Wednesday each week, committers merge all the issues that passed testing into the git production repository, and it becomes available for use on production systems via git, cvs, and download packages.


This SCRUM team generally works on the bugs important for improving the current stable release (STABLEBACKLOG).  Just before any major release they also work on backup/restore for that release.  This ensures a good familiarity with the code-base of the next release.


Initial staff: Andrew, Aparup, Dongsheng, Jerome, Rossi, Sam


# Selects one month's work (STABLEBACKLOGSPRINT) from the highest priority in STABLEBACKLOG
# Assigns X to them so they are visible/manipulable in Greenhopper as a Sprint, eg "Jan 2011", and also get assigned "Fix version" of the next minor release (eg 2.0.1).
# Meets with Product Owner to resolve any questions, agree on details
# Assigns work among team members
# Meets daily with Scrum Master to keep on track, resolve issues
# Develops solutions on local git installs
# Pushes solutions as issue-named branches (eg wip-MDL-12345) to a public repository (moodle.com ?)
# Publish suggested code via links in comments in Jira
# Conduct basic peer-reviews of code from other team members (direct communication, Jira, Crucible?)
# Rebase WIP branches and remove "wip-" from the name so it's called MDL-12345
# Create PULL requests for each issue or group of related issues for Integration review


==Stable maintenance cycles==


===Moodle HQ New Developments team===
Moodle releases regular updates of the stable version of the software to fix bugs and other issues.  Releases like 2.0.1, 2.0.2, 2.0.3 etc only include fixes based on the latest major release (2.0) and never any significant new features or database changes.


Initial staff: David,
At Moodle HQ there is a team of developers using a SCRUM-based process to work on these releases.


# Selects one month's work (DEVBACKLOGSPRINT) from the highest priority in DEVBACKLOG
===Triage team===
# Assigns X to them so they are visible/manipulable in Greenhopper as a Sprint, eg "Jan 2011", and also get assigned "Fix version" of the next major release (eg 2.1).
# Meets with Product Owner to resolve any questions, agree on details
# Extend/deepen specifications as necessary
# Assigns work among team members
# Develops solutions on local git installs
# Pushes work-in-progress solutions as issue-named branches (eg wip-MDL-12345) to a public repository (moodle.com, github.com ?)
# Publish suggested code via links in comments in Jira
# Conduct basic peer-reviews of code from other team members (direct communication, Jira, Crucible?)
# Rebase WIP branches and remove "wip-" from the name so it's called MDL-12345
# Create PULL requests for each issue or group of related issues for Integration review


Triagers evaluate new issues, making sure that they are labelled correctly.  One of the most important jobs they do is to identify issues that should be fixed in the stable branch.  These are added to the STABLE backlog with a priority ranging from "Trivial" up to "Blocker".


===External Developers===
Other larger issues go into the DEV backlog for consideration as part of the next major release.


# Develop whatever they like, however they like!
===Scrum team===
# Publish to public git repositories (github, gitorious, whatever)
# Create/find issues in Jira
# Suggest code via links in comments in Jira


Every three weeks, the Stable Scrum team takes a number of the most urgent issues from the STABLE backlog  (approx 30, but it varies) and starts working on solving them.  The team meets daily to discuss solutions and progress, as well as conducting peer reviews of code.  The team has a "Scrum master" to help everyone stay organised and to "unblock" any barriers to progress.  Everything is documented publically in the tracker.


Whenever a solution for an issue is finished, it is submitted into to the standard integration workflow process described above.


===Component maintainer role===


# Reviews and selects final solution for each issue in their control
# Possibly rebases, cleans up
# Create PULL requests for each issue or group of related issues for Integration review




==Major release cycles==


===Integration reviewer role===
Since Moodle 2.0, we have a policy of release major versions (eg 2.1, 2.2) every six months: in June and December.


Initial staff: Petr, Eloy, Sam
Each release can be different, but generally the cycles work like this:


Only these people have write access to the integration repository '''git.moodle.org/integration.git'''.
===Define roadmap===


# At set times per week, the Integration Review team goes through the current queue of “pull requests” (reviewers must not review/commit their own code)
The product owner (Martin Dougiamas) defines the likely roadmap based on community wishes, third-party developments and important issues within the existing code.  The DEV backlog is also a factor.
# Each new “pull request” is either accepted or rejected with comments.
# If accepted then the “pull request” is merged into the integration repository to Moodle branches and the “pull request” issue changes status to “QA review”
# If rejected then the reviewer marks the pull request as “Rejected” and the component maintainer is told why in the Jira issue


Sometimes these might be based on earlier features, sometimes they may be evauluating something developed by a third party, sometimes it might be something completely new.


===[[Development:Testing|QA testing role]]===


Initial staff: Helen, volunteers
===Development team===


# Go through the “QA review” issues in the “pull request” queue.
The DEV team (employed at Moodle HQ) work on major new features one or two at a time.  The aim is always to try and finish things completely before moving on to the next features, as this helps to avoid delays in delivering a release on time.
# Test the new features/fixes on automatically-built Moodle sites (or their own)
# Sign off on the fix and change the status of the “pull request” to “QA success”.
# If the test fails, comment and change the status of the “pull request” to “QA fail”


===Production maintainer role===
All this code goes through the standard weekly integration workflow described above.


Initial staff: Petr, Eloy
===QA Team===


Only these people have write access to the official production repository '''git.moodle.org/moodle.git''', and all commits come direct from the integration repository.
Finally, in the last month before the release, a feature freeze is called (no new features can be added) and our QA team performs a full functional test of Moodle features (manually, one at a time) to make sure there are no regressions caused by new features.


# At set times per week, go through the list of “QA fail” and “QA review” pull requests and make sure that code is not in the integration server, or that the Integration reviewer fixes them. All pull requests should be closed with “QA success” or “Rejected”.
The current list of functional tests is listed in the tracker in the [http://tracker.moodle.org/browse/MDLQA-1 MDLQA project].  This list of tests is extended on every release to include new developments.
# Push the integrated Moodle branches to the production server.
# Close the original Jira issues, say thanks and buy drinks.


==Commit messages==


Every commit should have a commit message to help other developers follow and review what is being changed. They are very helpful when tyring to understand a piece of code. The command <tt>git blame</tt> will tell you who last changed each line of code, and when they did it. Then you can look at the commit comments to tell you why that change was made.


A commit message should contain, in order:
* A couple of words indicating which part of moodle this change affects. For example 'forum backup', 'quiz attempting' or 'weblib'.
* A tracker issue id like MDL-12345. (If necessary, create an issue before committing, the only exception is for very minor typos.)
* A brief description of the problem you are fixing. (If you are feeling lazy, you may be able to get away with copying and pasting the issue summary from the tracker.)
* If it is not immediately obvious, a brief summary of why this change fixes the problem. If a longer description is necessary,  you may also want to add more information to the tracker issue, or Moodle Docs, but the code changes + commit message should provide enough clues to how the fix works on their own.


Here is an example of a commit message from a simple change:


user selection: MDL-17072 Polishing the role assign page. Do processing before print_header in line with best practice.
==New feature development==


Actually, the ideal commit message would be a bit shorter than that. It is good if the first three items (area of Moodle, tracker id and brief description) will fit in the subject line of an email message, so the first line should be shorter than about 70 characters if possible.
Major new features in Moodle usually should go through the following process.


Here is an example from a bigger commit (would it have been better if this could have been done as several smaller commits?):
===Specification===
role overrides: MDL-17070 Improve override roles page to match the recent usability improvements on the assign page.
Including:
MDL-11529 When assigning/overriding roles, the dropdown for switching to
another role should have a number in brackets
MDL-16549 Should not be able to edit the permission associated with
moodle/site:doanything on any role.


Here is an example of a commit (from a while ago, which is why the issue number is in the wrong place) with some explanation of the change:
The developer should create a detailed spec (here in Moodle Docs) outlining their goals for the development and their design for meeting those goals.  The more detail the better.


accesslib: get_user_by_capability() - Handle complex rolecap resolution
You should also create an issue in the tracker (linking to your docs) to keep track of the project status.
With this patch, get_user_by_capability() can handle the cases where
users have multiple role assignments to the same course, and PREVENTs
and PROHIBITs affect the rolecaps of this course.
Without stored procedures we cannot resolve this entirely on the
server side - so in the complex cases we do as much as we can on SQL,
and post-process the data on the PHP side, including SQL-style
pagination.
MDL-12452


==Workflow in tracker==
===Community consultation===


Proposed new workflow for MDL issues.  
Get the community involved in looking at the spec to see if it meets their needs and to get further feedback.  The forums on moodle.org are good for this, but you could also blog/tweet about it etc.


[[Image:Workflow.jpg]]
You probably also want to talk with HQ core developers to make sure the ideas make sense, and possibly get some review on database design, architecture and so on.


==Scheduling==
===Develop the code using git===


Here are some suggested scheduling times for different parts of the process.
Develop your code on an open git repository, like github.com.  That enables people to see your code and to help you as it develops.  Testers and early adopters also have the opportunity to try it early in the process and give you more valuable feedback.


===Integration review===
It is essential that your code follows the [[Development:Coding|Moodle Coding Guide]]


* Monday - Review team divides PULL backlog and reviews all code -> Accept/Reject, commit to integration repo
===Submit the code for integration===
* Tuesday - QA testing on all code accepted on integration repository
* Wednesday - code pushed to main repository


===Stable scrum sprints===
This is much the same as for any Moodle code.  See the information about the integration workflow above.


* Day 1, Sprint planning meeting, choosing and assigning high-priority tasks from the backlog to the sprint
* Day 2-19, daily stand-up meeting, developing solutions, add requests to the PULL backlog
* Day 20, Sprint Review meeting, Sprint Retrospective meeting




Latest revision as of 05:32, 27 April 2011

This document summarises the various development processes used in developing Moodle. There are four main processes that overlap.

Here are some terms we'll use in this document:

  • STABLE = the next stable minor version of Moodle (eg 2.0.1, 2.0.2 ... etc)
  • STABLE backlog = the queue of bugs to be fixed in STABLE (eg 2.0.x)
  • DEV = the next major version of Moodle (eg 2.1, 2.2, 2.3 ... etc)
  • DEV backlog = everything to be looked at for upcoming major versions (eg 2.X)


Integration workflow in the tracker

The Moodle tracker keeps track of the status of all bug fixes and new features.

We use a workflow that ensures that new code receives multiple reviews by different people before it is included into the core Moodle code.

Workflow.jpg

A number of roles make this work:

Users

Users report bugs and make feature requests directly in the tracker, by creating new issues with a summary and a description.

Developers

Developers work on the issues in the tracker to specify solutions and write code that implements these solutions. They will often ask other developers to "peer review" their code in the early stages to avoid problems later on.

While many of the developers work for Moodle.com, a large number are part of the global development community around Moodle.

Component leads

Component leads are developers with some responsibility for particular components (plugins or modules) in Moodle. They have the authority to decide that a particular fix is suitable and complete enough to be considered for integration in Moodle core, by adding their git branches to the integration queue. This can happen at any time.

Integrators

On Monday of each week, the integration team (a small team of senior developers employed by Moodle HQ) conducts a code-level review of all issues in the integration queue. This is often called the "pull" process. If the fix is judged appropriate they will integrate the code into our git integration repository for further testing and it gets added to the testing queue.

If they find problems they reject the issue and send it back to the developer for further work.

Testers

On Tuesday each week, testers look at all the issues in the testing queue, trying each fix and feature to make sure that it does actually fix the problem it was supposed to, and that there are no regressions.

If they find problems they reject the issue and integrators may remove it from the integration repository and push it back to the developer for further work.

Committers

On Wednesday each week, committers merge all the issues that passed testing into the git production repository, and it becomes available for use on production systems via git, cvs, and download packages.



Stable maintenance cycles

Moodle releases regular updates of the stable version of the software to fix bugs and other issues. Releases like 2.0.1, 2.0.2, 2.0.3 etc only include fixes based on the latest major release (2.0) and never any significant new features or database changes.

At Moodle HQ there is a team of developers using a SCRUM-based process to work on these releases.

Triage team

Triagers evaluate new issues, making sure that they are labelled correctly. One of the most important jobs they do is to identify issues that should be fixed in the stable branch. These are added to the STABLE backlog with a priority ranging from "Trivial" up to "Blocker".

Other larger issues go into the DEV backlog for consideration as part of the next major release.

Scrum team

Every three weeks, the Stable Scrum team takes a number of the most urgent issues from the STABLE backlog (approx 30, but it varies) and starts working on solving them. The team meets daily to discuss solutions and progress, as well as conducting peer reviews of code. The team has a "Scrum master" to help everyone stay organised and to "unblock" any barriers to progress. Everything is documented publically in the tracker.

Whenever a solution for an issue is finished, it is submitted into to the standard integration workflow process described above.



Major release cycles

Since Moodle 2.0, we have a policy of release major versions (eg 2.1, 2.2) every six months: in June and December.

Each release can be different, but generally the cycles work like this:

Define roadmap

The product owner (Martin Dougiamas) defines the likely roadmap based on community wishes, third-party developments and important issues within the existing code. The DEV backlog is also a factor.

Sometimes these might be based on earlier features, sometimes they may be evauluating something developed by a third party, sometimes it might be something completely new.


Development team

The DEV team (employed at Moodle HQ) work on major new features one or two at a time. The aim is always to try and finish things completely before moving on to the next features, as this helps to avoid delays in delivering a release on time.

All this code goes through the standard weekly integration workflow described above.

QA Team

Finally, in the last month before the release, a feature freeze is called (no new features can be added) and our QA team performs a full functional test of Moodle features (manually, one at a time) to make sure there are no regressions caused by new features.

The current list of functional tests is listed in the tracker in the MDLQA project. This list of tests is extended on every release to include new developments.



New feature development

Major new features in Moodle usually should go through the following process.

Specification

The developer should create a detailed spec (here in Moodle Docs) outlining their goals for the development and their design for meeting those goals. The more detail the better.

You should also create an issue in the tracker (linking to your docs) to keep track of the project status.

Community consultation

Get the community involved in looking at the spec to see if it meets their needs and to get further feedback. The forums on moodle.org are good for this, but you could also blog/tweet about it etc.

You probably also want to talk with HQ core developers to make sure the ideas make sense, and possibly get some review on database design, architecture and so on.

Develop the code using git

Develop your code on an open git repository, like github.com. That enables people to see your code and to help you as it develops. Testers and early adopters also have the opportunity to try it early in the process and give you more valuable feedback.

It is essential that your code follows the Moodle Coding Guide

Submit the code for integration

This is much the same as for any Moodle code. See the information about the integration workflow above.

Retrieved from "https://docs.moodle.org/27/en/index.php?title=Development:Process&oldid=83051"
Powered by MediaWiki