Development:Bug triage: Difference between revisions

Latest revision as of 13:47, 16 February 2011

Note: This page is for analysing and developing a bug triage process. It is a work-in-progress. Comments or suggestions are welcome! Please use the page comments.

Triage is medical term referring to the process of prioritizing patients based on the severity of their condition so as to maximise benefit (help as many as possible) when resources are limited.

Bug triage is a process where tracker issues are sorted and categorised. It should help ensure we appropriately manage all reported issues - bugs as well as improvements and feature requests.

The triage process

General

Has the issue been reported previously?

Duplicate issues should be closed with no fix version set.

Are the correct components selected?
If necessary, request more information, such as debug messages for errors
If applicable, for example the issue covers more than one component, add watchers

Priority

Security issues and anything breaking badly (error/exception) => blocker
Anything leading to wrong results => critical
Anything else => major (apart from minor/trivial tweaks)

Note: Improvements/new features cannot be blockers (in triaging process, though the dev team can raise them later)

Assignee

Generally be the main component lead
Otherwise the developer currently assigned to most issues for that component (click on the component link and it will show current assignees and then click on "resolved" to see who has been working on it recently)
Otherwise leave as moodle.com (for the team to grab later as the sprint progresses)

Fix version

If the issue is a bug in the current stable version (eg 2.0), assign "Fix version" to STABLE backlog
If the issue is related to backup and restore, assign "Fix version" to STABLE backlog with high priority
If the issue is a bug in the current stable version requiring database changes, assign "Fix version" to DEV backlog
If the issue is a feature request, assign "Fix version" to DEV backlog

If the issue already has a fix version set by the component lead, it should be left unchanged.

For meta issues, the fix version may be set as 2.x, as generally only subtasks of meta issues are included in sprints.

Labels

Issues are labelled triaged to indicate that all the above points have been covered. A comment may be added explaining the reason for any changes.

Issues with proposed fixes are labelled patch so that they can be found easily and given attention.

Triage filter

The filter To triage (Moodle-all) includes:

Issues with assignee to review (empty/nobody/moodle.com without triage label)
Issues with fix version empty or set to some released version
Issues not included in a backlog and not yet labelled as triaged

Triaging priorities

By priority - aim for blockers and critical issues to be reduced to 0
Recent bugs
Whatever is of special interest at a given time e.g. for DEV sprint forumng, survey, lesson and gradebook

Example query for searching for performance-related issues:

project = "Moodle" AND
status in (open, reopened, "In Progress") AND
"Affected Branches" ~ "MOODLE_20_STABLE" AND (
   component in (Performance) OR
   labels in ("performance") OR
   text ~ slow OR
   text ~ speed OR
   text ~ performance OR
   text ~ profiling OR
   text ~ benefit OR
   text ~ throughput)
ORDER by priority desc

@@ Line 1: / Line 1: @@
-This page is for analysing and developing a Moodle issue triage process. This process is STILL UNDER CONSTRUCTION.
+<p class="note">'''Note:''' This page is for analysing and developing a bug triage process. It is a work-in-progress. ''Comments or suggestions are welcome!'' Please use the [[{{TALKPAGENAME}}|page comments]].</p>
 Triage is medical term referring to the process of prioritizing patients based on the severity of their condition so as to maximise benefit (help as many as possible) when resources are limited.
-Moodle issue triage is a process where issues are sorted and categorised with the intent of maximising benefit to Moodle given limited resources.  It should help us act on those issues that require immediate attention (e.g. high priority, affects most users, is a security issue) while relegating issues deemed less critical to a lessor status, to be actioned at a later date.
+Bug triage is a process where tracker issues are sorted and categorised. It should help ensure we appropriately manage all reported issues - bugs as well as improvements and feature requests.
-Triaging issues should help ensure we appropriately manage all reported issues (bugs as well as feature requests).
-The following white paper assumes a 2-step process: (1) quality review of information in the Tracker, and (2) triage
+==The triage process==
-The primary benefit of a 2-step process is to allocate resources appropriately: team members with greater product knowledge can focus on triage while those with less knowledge can focus on Tracker quality assurance (while hopefully building product knowledge).
+===General===
+* Has the issue been reported previously?
+Duplicate issues should be closed with no fix version set.
-==Tracker Quality Review==
+* Are the correct components selected?
-Each Tracker issue must be in a "fit state", so that an informed decision can be made about it.  This is a quality review of the information entered into Tracker.  Every new issue entered into the Tracker should be reviewed.
+* If necessary, request more information, such as debug messages for errors
+* If applicable, for example the issue covers more than one component, add watchers
-Here is a starting point for Tracker QA:
+===Priority===
-*Does the Summary accurately reflect the issue?
+* Security issues and anything breaking badly (error/exception) => blocker
-*Is there enough information for a developer or a tester to reproduce the problem.
+* Anything leading to wrong results => critical
-*Are fields completed adequately?
+* Anything else => major (apart from minor/trivial tweaks)
-*Has it been categorised correctly (priority is an obvious example)
-*Has it been assigned to the appropriate tester/developer?
-Action points:
+Note: Improvements/new features cannot be blockers (in triaging process, though the dev team can raise them later)
-*Identify a filter to ensure every new issue is reviewed on a periodic basis.  Is the nightly?  Weekly?
-*Identify a person/team to review new issues for quality.
-*Identify a way to know that each issue has been reviewed.
-==Triage process==
+===Assignee===
-Triage should take place against issues that have been verified.
-Action item:  identify a person/team with sufficient product knowledge to make and informed call on newly logged issues.
+* Generally be the main component lead
+* Otherwise the developer currently assigned to most issues for that component (click on the component link and it will show current assignees and then click on "resolved" to see who has been working on it recently)
+* Otherwise leave as moodle.com (for the team to grab later as the sprint progresses)
-==Triage filters==
+===Fix version===
-*[http://tracker.moodle.org/secure/IssueNavigator.jspa?mode=hide&requestId=10544 1.9.x Reported (Need Triage)] - These are bugs that are reported as being in 1.9 stable branch but appear not to be fixed yet. They need to be looked at and given a fix version or closed. Note that many of these may not be relevant, or may be miscategorised, so they need to be processed.
+* If the issue is a bug in the current stable version (eg 2.0), assign "Fix version" to STABLE backlog
+* If the issue is related to backup and restore, assign "Fix version" to STABLE backlog with high priority
+* If the issue is a bug in the current stable version requiring database changes, assign "Fix version" to DEV backlog
+* If the issue is a feature request, assign "Fix version" to DEV backlog
-*[http://tracker.moodle.org/secure/IssueNavigator.jspa?mode=hide&requestId=10231 Recent unresolved bugs that need triage] - These bugs need checking to make sure they are assigned to the right person, category etc.
+If the issue already has a fix version set by the component lead, it should be left unchanged.
-==Fix versions==
+For meta issues, the fix version may be set as 2.x, as generally only subtasks of meta issues are included in sprints.
-Bugs are removed from the triage list by giving them a fix version or closing them. Confirmed bugs should be given the next unreleased version as the fix version, for example a confirmed bug in 1.9 should have a 1.9.1 fix version.
+===Labels===
-==Adding watchers==
+Issues are labelled ''triaged'' to indicate that all the above points have been covered. A comment may be added explaining the reason for any changes.
-If assistance is required with a bug, additional watchers may be added then a comment added. If documentation is required, Helen should be added as a watcher, then a comment saying "this requires documentation" added.
+Issues with proposed fixes are labelled ''patch'' so that they can be found easily and given attention.
+==Triage filter==
+The filter [http://tracker.moodle.org/secure/IssueNavigator.jspa?mode=hide&requestId=11424 To triage (Moodle-all)] includes:
+* Issues with assignee to review (empty/nobody/moodle.com without triage label)
+* Issues with fix version empty or set to some released version
+* Issues not included in a backlog and not yet labelled as triaged
+==Triaging priorities==
+# By priority - aim for blockers and critical issues to be reduced to 0
+# Recent bugs
+# Whatever is of special interest at a given time e.g. for DEV sprint forumng, survey, lesson and gradebook
+Example query for searching for performance-related issues:
+ project = "Moodle" AND
+ status in (open, reopened, "In Progress") AND
+ "Affected Branches" ~ "MOODLE_20_STABLE" AND (
+    component in (Performance) OR
+    labels in ("performance") OR
+    text ~ slow OR
+    text ~ speed OR
+    text ~ performance OR
+    text ~ profiling OR
+    text ~ benefit OR
+    text ~ throughput)
+ ORDER by priority desc
 ==See also==
+*[[Tracker]]
+*[[Development:Process]]
 *[[Development:Weekly Code Review]]
 *[[Development talk:Bug triage]]
 [[Category:Developer|Bug triage]]
+[[Category:Quality Assurance]]

Documentation