Bug triage

Note: This page is for analysing and developing a bug triage process. It is a work-in-progress. Comments or suggestions are welcome! Please use the page comments.

Triage is medical term referring to the process of prioritising patients based on the severity of their condition so as to maximise benefit (help as many as possible) when resources are limited.

Bug triage is a process where tracker issues are screened and prioritised. Triage should help ensure we appropriately manage all reported issues - bugs as well as improvements and feature requests.

The triage process

Initial screening

The following aspects should be checked first to ensure the issue can be worked on.

Has the issue been reported previously?

If so, link to a duplicate issue and close the newly reported issue with no fix version set. Encourage the reporter to search before reporting. If a newer issue has a patch or more voters/watchers, consider closing the older issue. Checking for duplicates first will save you having to check the rest of the issue.

Does the problem seem rational?

If not, then the problem may simply be an misunderstanding on the part of the reporter. It might be a problem exclusive to the reporter's server setup. If you can replicate the problem quickly, do so. If you can't replicate the problem, ask the reporter to attempt to replicate the problem on demo.moodle.net. If the problem seems persistent but strange, consider asking a developer with experience working in the area to consider the problem and determine if it could be a real problem.

Can the problem be replicated?

There should be sufficient information in the issue so that someone else could replicate it. Ideally this would include:

• replication steps (with expected results and actual results),
• error messages (if applicable) and
• screenshots (if applicable).

If more information is needed, do not complete the triage. Instead, watch the issue and refer it back to the reporter. If the reporter does not respond with more detail in a reasonable amount of time (say one week), close the issue as incomplete, but keep a watch on the issue.

Here is an example comment that could be used to elicit further information from a reporter.

Thanks for reporting this.

Could you please add more information to your report such as replication instructions, error messages and screenshots. If you are able to suggest a workaround, that would be immediately helpful to others experiencing this problem. If you can determine the cause of the problem or even determine a solution, that will be very valuable.

Components

Are the correct components selected?

Listing components correctly is important as they are the primary variable used for searching for issues.

If the issue is listed with more than one component or if you change the components, the default assignee might not be set correctly.

If the issue covers more than one component determine the main component and ensure the assignee is set in relation to that component. Add component leads of other components as watchers if there is a significant overlap between components.

Security level

Is the issue a security issue?

If so, this should be determined as soon as possible.

Issues are often raised with a tentative "Could be..." security level. Ideally this should be set to "none", "minor" or "serious" as soon as possible to control the visibility of the issue.

Minor	problem could inappropriately reveal personal information to existing site users problem could give existing users an inappropriate level of access
Serious	problem could inappropriately reveal personal information to non-users problem could be exploited to attack a site, potentially altering the site or denying access

Priority

• Security issues should be blockers.
• Backup/restore issues should have a high priority.
• Other issues should follow the Tracker guide.

Note: Improvements/new features cannot be blockers and should usually be left with a Minor priority (though the dev team can raise the priority later)

Fix version

• If the issue is a bug in the current stable version, assign to STABLE backlog
• If the issue is a bug in the current stable version requiring database changes, assign to DEV backlog
• If the issue is an improvement, generally assign to DEV backlog unless the fix is very minor (not involving DB changes)
• If the issue is a new feature request, assign to DEV backlog

If the issue already has a fix version set by the component lead, it should be left unchanged.

Assignee

• Generally be the main component lead
• Otherwise the developer currently assigned to most issues for that component (click on the component link and it will show current assignees and then click on "resolved" to see who has been working on it recently)
• Otherwise leave as moodle.com (for the team to grab later as the sprint progresses)

Labels

Issues are labelled triaged to indicate that all the above points have been covered. A comment may be added explaining the reason for any changes.

Issues with proposed fixes are labelled patch so that they can be found easily and given attention - although consider whether moving the issue to the 'Waiting for peer review' state in the workflow might be more appropriate.

There is a list of standard labels that can be used in this field. Other labels should be removed; if necessary, they could be copied to the description.

Testing instructions

The Testing instructions field is used during testing, after an issue has been resolved.

Often issue reporters will include reproduction steps in this field. It is good that this information is provided by the reporter, but it should be moved to the description unless it is seen to constitute an appropriate set of testing instructions.

Gratitude and encouragement

After triaging many issues, it's easy to lose sight of the fact that the reporter has contributed their time and energy to report an issue for the benefit of the community.

It is easy to become defensive of Moodle if reports are seen as criticism (and sometimes reporters may use phrasing that suggests this), however the triagers response must always be one of sincere gratitude.

It is also important to encourage reporters to continue being involved with the issue after it is triaged. We must not give the sense that we are taking the issue ownership away from the reporter. Instead the reporter should be encouraged to discover the cause of the problem and present a solution; this is appropriate in an open-source project. It is amazing that such a challenge can lead to a sense of purpose for the reporters.

Triage filter

The filter To triage (Moodle-all) includes:

• Issues with assignee to review (empty/nobody/moodle.com without triage label)
• Issues with fix version empty or set to some released version
• Issues not included in a backlog and not yet labelled as triaged

Triaging priorities

1. By priority - aim for blockers and critical issues to be reduced to 0
2. Recent bugs
3. Whatever is of special interest at a given time e.g. for DEV sprint forumng, survey, lesson and gradebook

Example query for searching for performance-related issues:

project = "Moodle" AND
status in (open, reopened, "In Progress") AND
"Affected Branches" ~ "MOODLE_20_STABLE" AND (
   component in (Performance) OR
   labels in ("performance") OR
   text ~ slow OR
   text ~ speed OR
   text ~ performance OR
   text ~ profiling OR
   text ~ benefit OR
   text ~ throughput)
ORDER by priority desc

See also

• Tracker guide
• Process
• Talk:Bug triage