Difference between revisions of "Bug triage"

Jump to: navigation, search
(Adding note about addon-candidate label)
(Initial screening: Is it a usability issue?)
 
(34 intermediate revisions by 7 users not shown)
Line 4: Line 4:
 
Bug triage is a process where tracker issues are screened and prioritised. Triage should help ensure we appropriately manage all reported issues - bugs as well as improvements and feature requests.
 
Bug triage is a process where tracker issues are screened and prioritised. Triage should help ensure we appropriately manage all reported issues - bugs as well as improvements and feature requests.
  
 +
Triage initially happens shortly after the issue was reported but it can be repeated later if the initial assumptions were wrong, issue was resolved otherwise, affected versions need updating or there are other reasons to review the issue.
  
==The triage process==
+
==Get involved==
  
===Initial screening===
+
Anybody can do triage in form of correcting the components and/or affected versions, linking to related issues, and of course commenting asking for clarification, confirming bug, redirecting to forum, etc. Users in ''jira-developers'' and ''moodle-triage'' groups can edit any issue, ''jira-users'' can comment on any issue or edit issues they reported. Please see MDLSITE-3592 if you are not a developer but would like to help with triage process.
  
The following aspects should be checked first to ensure the issue can be worked on.
+
Adding ''triaged'' label and placing the issue on the backlog should only be done by component lead or HQ developer. At the same time other users can remove ''triaged'' label from the old issues or replace it with ''triaging_in_progress'' if they want to request an additional triage.
  
; Has the issue been reported previously?
+
==The triage process==
: If so, link to a duplicate issue and close the newly reported issue with no fix version set. Encourage the reporter to search before reporting. If a newer issue has a patch or more voters/watchers, consider closing the older issue. Checking for duplicates first will save you having to check the rest of the issue.
 
  
Here is a comment that can be used when closing a duplicate.
+
===Initial screening===
<p class="note">
 
Thanks for your report.<br /><br />
 
It seems you're not the only one to have come across this bug, as it's been reported previously - see MDL-xyz. I'm going to close this issue now so we can focus on MDL-xyz. Please watch, vote ir comment on MDL-xyz if there is any additional information you can provide.
 
</p>
 
  
; Does the problem seem rational?
+
First of all, identify the issues that should be closed or placed under investigation. Ask the following questions:
: If not, then the problem may simply be an misunderstanding on the part of the reporter. It might be a problem exclusive to the reporter's server setup. If you can replicate the problem quickly, do so. If you can't replicate the problem, ask the reporter to attempt to replicate the problem on demo.moodle.net. If the problem seems persistent but strange, consider asking a developer with experience working in the area to consider the problem and determine if it could be a real problem.
 
; Can the problem be replicated?
 
: There should be sufficient information in the issue so that someone else could replicate it. Ideally this would include:
 
:* replication steps (with expected results and actual results),
 
:* error messages (if applicable) and
 
:* screenshots (if applicable).
 
: If more information is needed, do not complete the triage. Instead, watch the issue and refer it back to the reporter. If the reporter does not respond with more detail in a reasonable amount of time (say one week), close the issue as incomplete, but keep a watch on the issue.
 
  
Here is an example comment that could be used to elicit further information from a reporter.
+
* '''Is the issue a request for support/help?''' If so, the reporter should be directed to the forums to seek help and the issue should be closed as "Not a bug". Sometimes improvement requests can be phrased as a question, though; if this is the case, ask the reporter to reword the description to describe an improvement.
<p class="note">
+
* '''Have the reporter mistaken the Moodle Tracker with their own support desk?''' Sometimes people mistake the Moodle Tracker as a place to request help about their own Moodle instance, often about logging in. We need to refer the user to their instance administrators and close the issue as "Not a bug".
Thanks for reporting this.<br /><br />
+
* '''Has the issue been reported previously?''' If so, link to a duplicate issue and close the newly reported issue as a "Duplicate" with no fix version set. Encourage the reporter to search before reporting. If a newer issue has a patch or more voters/watchers, consider closing the older issue. Checking for duplicates first will save you having to check the rest of the issue. See [[Tracker tips]] for help with effective search of tracker.
Could you please add more information to your report such as replication instructions, error messages and screenshots. If you are able to suggest a workaround, that would be immediately helpful to others experiencing this problem. If you can determine the cause of the problem or even determine a solution, that will be very valuable.
+
* '''Does the problem affect only unsupported versions?''' If so, the issue should be closed using "Fixed" (preferred as it sounds better) when the issue is resolved in current versions or "Not a bug" when the issue has disappeared due to changes leading to current versions. See [Releases] to find currently supported versions
</p>
+
* '''Did the problem arise because of mistake in documentation or lack thereof?''' If it appears that the reporter does not understand a particular feature in Moodle and the documentation is lacking, ask the reporter where would he expect to find documentation about it. Then simply edit the relevant pages in the documentation wiki and close the issue. If required change is significant, add ''Documentation'' component and ''docs_required'' label.
 +
* '''Is the problem a language string change?''' Language string problems can be corrected by [[Contributing a translation|contributing a translation]] or by contacting the language pack maintainer as listed in the [https://lang.moodle.org/local/amos/credits.php Translation credits]. English language string typo fixes and suggested improvements can be [[Improving English language strings|contributed to the English (fixes) (en_fix) language pack]] or given the component 'Language' for fixing by the Language component lead. Such issues should be closed in the Tracker using "Deferred".
 +
* '''Is it a usability issue?''' If so, add the component "Usability" in addition to the component(s) specifying the area of Moodle.  
 +
* '''Was the problem caused by additional code or 3rd party plugins?''' If you can identify the plugin, move the issue to the respective component of CONTRIB project. Otherwise comment and close as "Not a bug".
 +
* '''Can this be implemented as a plugin?''' And maybe the plugin already exists in the plugins directory. Explanation should be given to the reporter that Moodle provides the framework but does not work on any possible plugin. Add label "addon_candidate" but do not close the issue. This can also apply to the requests to significantly redesign existing plugins and it would be more preferable to create a new alternative plugin.
 +
* '''Does the problem seem rational?''' If not, then the problem may simply be an misunderstanding on the part of the reporter. It might be a problem exclusive to the reporter's server set-up. If you can replicate the problem quickly, do so. If you can't replicate the problem, ask the reporter to attempt to replicate the problem on http://demo.moodle.net. If the problem seems persistent but strange, consider asking a developer with experience working in the area to consider the problem and determine if it could be a real problem.
 +
* '''Can the problem be replicated?''' If not, or information on the issue is insufficient, ask the reporter to add error messages, screenshots, environment information (OS, web server, browser) and exact replication instructions
  
If the report clearly relates to a problem with the reporter's server or local set-up, you could use the following statement. (It's easy to slip into help-desk mode, but that is not the purpose of the Tracker.)
+
As a result of initial screening up to 20% of new issues may be closed. When closing the issues make sure to set the correct resolution and write a polite comment with explanation, refer to the templates below. If you have doubts, ask the questions and always add label [https://tracker.moodle.org/issues/?jql=labels%20in%20%28triaging_in_progress%29 triaging_in_progress]. Subscribe to the filter [https://tracker.moodle.org/issues/?jql=labels%20in%20%28triaging_in_progress%29%20AND%20project%20%3D%20MDL%20AND%20resolution%20%3D%20Unresolved%20AND%20Participants%20%3D%20currentUser%28%29%20AND%20updatedDate%20%3C%20-30d My old issues in triage] and you will receive notifications after 30 days of inactivity on such issues. See [[Tracker tips]] about how to subscribe to filter. Very often the reporter never follows up on their own issues and this is a good way to find such issues and ping reporter again or make a final decision about closing.
<p class="note">
 
It sounds like this is a problem with your server set-up or the way you have configured Moodle. It doesn't sound like this is a problem in the Moodle code.<br /><br />
 
You might find help on the [http://moodle.org/mod/forum/view.php?id=28 Installation forum].
 
</p>
 
  
===Components===
+
===Confirming the issue===
; Are the correct components selected?
 
: Listing components correctly is important as they are the primary variable used for searching for issues.
 
:* If the issue is listed with more than one component or if you change the components, the default assignee might not be set correctly.
 
:* If the issue covers more than one component determine the main component and ensure the assignee is set in relation to that component. Add component leads of other components as watchers if there is a significant overlap between components.
 
  
===Security level===
+
When you confirm that issue is indeed a bug or a reasonable improvement request that was not reported previously, make sure that the following is accurate:
  
; Is the issue a security issue?
+
* '''Security level.''' Security level must be set as soon as possible if the reported bug discloses vulnerability in Moodle that can be exploited to access information without appropriate level, create an attack on the site, embed XSS or forge a request. In some rare cases Improvements may be also marked as security
: If so, this should be determined as soon as possible.
+
* '''Summary.''' Summary of the issue should clearly describe the problem or improvement area, rephrase such summaries as "Some improvements in xxx" or "Error in Moodle", etc.
: Usually security issues should be bugs; improvements that improve security need not have a security level set (unless they reveal a weakness in Moodle, in which case they should be a bug).
+
* '''Issue Type.''' The following issue types are used in Moodle:
: Issues are often raised with a tentative "Could be..." security level. Ideally this should be set to "none", "minor" or "serious" as soon as possible to control the visibility of the issue.
+
** Bug - represents an actual bug and should be fixed in all supported versions. Very often when reporter expects something to be better than it actually is they call it a bug when it's in fact an improvement.
 +
** Improvement - improvement to existing functionality. If addressed, will be integrated in the following major release only
 +
** New feature - completely new feature, also will not be applied to the released versions (unless implemented as a plugin and submitted to plugins directory)
 +
** Task - usually created by developers themselves and can not be classified as Bug or Improvement, for example, adding automated tests, improving documentation, etc.
 +
** Epic - created by HQ developers or component leads to collect together issues that represents parts of one project. META issues and sub-tasks should not be used any more
 +
* '''Priority.''' Some reporters over-state an issue's priority. Some reporters don't know they can set a priority. Priority is used as one of the criteria when sorting issues in the backlog, so it should reflect the position of this issue comparing to the others. Usually Improvements have Minor or Major priority and Bugs can have any priority up to Blocker. Priority levels have specific criteria. Please see [https://docs.moodle.org/dev/Tracker_guide#When_editing_an_issue the Tracker guide]
 +
* '''Component/s.''' Listing components correctly is important as they are the primary variable used for searching for issues, also adding a component automatically include the component lead as a watcher. Issue may have several components when needed
 +
* '''Affects version.''' This field should include one or more released *and supported* versions of Moodle that are affected by the issue, with the following exceptions:
 +
** The issue is a bug in code that is present in the Master branch only, in which case the next major version should be used. (The next major version should not be used in conjunction with previous released versions, this won't make sense later.)
 +
** The issue is a new feature and is unrelated to any existing code in Moodle, in which case the 'Future dev' version should be used.
 +
* '''Labels.'''
 +
** Remove functionality tags that some reporters add as labels, only  [https://docs.moodle.org/dev/Tracker_issue_labels standard] labels or partner-specific labels are used in Moodle
 +
** Issues with proposed fixes are labelled ''patch'' so that they can be found easily and given attention. When this is the case, consider whether moving the issue to the 'Waiting for peer review' state in the workflow might be more appropriate
 +
** Add ''addon_candidate'' label if the functionality can be implemented as a plugin;
 +
** If you are component lead or HQ developer you may also add ''triaged'' label to indicate that triage process is completed. Use it only when the issue has actually been added to the backlog
  
{|-
+
''Note that "Fix version" is no longer used during triage. If you are in "moodle-triage" group, you can use the '''Triage screen''' to edit only aforementioned fields, see MDLSITE-3592.''
| valign=top | '''Minor'''
 
| valign=top |
 
* problem could inappropriately reveal personal information to existing site users
 
* problem could give existing users an inappropriate level of access
 
|-
 
| valign=top | '''Serious'''
 
| valign=top |
 
* problem could inappropriately reveal personal information to non-users
 
* problem could be exploited to attack a site, potentially altering the site or denying access
 
|}
 
  
===Priority===
+
When commenting on the issue give more details on replication, environment or testing. Ask questions, add watchers, modify description if needed. Link to as many related issues as possible. Do everything that will make the issues scope more clear and attract opinions, discussions and patches.
  
; Is the priority level reasonable and representing the issues real priority in respect to the community?
+
Don't forget to show '''Gratitude and encouragement'''.  After triaging many issues, it's easy to lose sight of the fact that the reporter has contributed their time and energy to report an issue for the benefit of the community.
: Some reporters over-state an issues priority. Some reporters don't know they can set a priority.
 
:* Security issues should be blockers.
 
:* Backup/restore issues should have a high priority.
 
:* Other issues should follow the [[Tracker guide]].
 
  
Note: Improvements/new features cannot be blockers and should usually be left with a Minor priority (though the dev team can raise the priority later)
+
It is easy to become defensive of Moodle if reports are seen as criticism (and sometimes reporters may use phrasing that suggests this), however the triagers response must always be one of sincere gratitude.
  
===Affects version===
+
It is also important to encourage reporters to continue being involved with the issue after it is triaged. We must not give the sense that we are taking the issue ownership away from the reporter. Instead the reporter should be encouraged to discover the cause of the problem and present a solution; this is appropriate in an open-source project. It is amazing that such a challenge can lead to a sense of purpose for the reporters.
  
; Does the Affects version make sense now and into the future?
+
=== Following up on issues ===
: This field should include one or more '''released''' versions of Moodle that are affected by the issue, with the following exceptions:
 
:* The issue is a ''bug'' in code that is present in the Master branch only, in which case the next major version should be used. (The next major version should not be used in conjunction with previous released versions, this won't make sense later.)
 
:* The issue is a new feature and is unrelated to any existing code in Moodle, in which case the 'Future dev' version should be used.
 
: If the reported Affects version is unsupported, consider whether the issue could affect currently supported versions. In some cases it is simple close such an issue when a component has been replaced in a recent version, but otherwise this might require a quick replication test in a supported version. If the issue only affects unsupported versions, the issue should be closed as 'Won't fix'.
 
  
The following is a comment that can be used when closing an issue that affects only unsupported versions.
+
Tracker is set up by default so as soon as you comment on any issue or edit it, you become an automatic watcher and any following change on the issue will be emailed to you.  
<p class="note">
 
Thanks for reporting this.<br /><br />
 
I'm closing this issue because I believe it affects only unsupported versions of Moodle. This issue will remain here in case other users have the same problem.<br /><br />
 
If you haven't already done so, I encourage you to upgrade to a supported version.
 
</p>
 
  
===Fix version===
+
If you have encouraged the reporter well, they may '''submit a patch''' or somebody else may do it. Make sure that the ''patch'' label is added or issue is sent to peer review. On the contrary, if ''patch'' label was added by somebody else but you clearly see that patch is far from ready, remove the label and leave a comment explaining it.
  
; Is the Fix version set correctly?
+
Users may also comment with additional details, screenshots, replicating instructions. It may happen that the issue gets re-evaluated and priority or summary changed.
: When an issue is reported, it can be added to a backlog of work that a developer at Moodle HQ can handle. That does not preclude other developers, who are not working at Moodle HQ, from working on the issue.
 
: The Fix version will later become the version of Moodle that the issue is fixed in, but that value should only be set at the time of integration.
 
: In the Moodle project (MDL issues), the Fix version can be set to the BACKEND backlog, FRONTEND backlog or no backlog, depending on the primary component of the issue. Being on a backlog does not ensure work will be completed, equally not being on a backlog does not mean work will not be done, it's just a helpful way of allocating work to devs at Moodle HQ.
 
{| class="nicetable"
 
|-
 
! BACKEND
 
! FRONTEND
 
! ''Handled in a different way''
 
|-
 
| valign="top" |
 
* Administration
 
* Authentication
 
* Automated functional testing
 
* Backup
 
* Caching
 
* Cohorts
 
* Database SQL/XMLDB
 
* Enrolments
 
* Events API
 
* Files API
 
* Gradebook
 
* Grading methods
 
* Groups
 
* Installation
 
* Libraries
 
* Licensing
 
* Logging
 
* Messages
 
* Performance
 
* phpdoc
 
* Portfolio API
 
* Repositories
 
* Roles / Access
 
* RSS
 
* Tags
 
* Unicode
 
* Unit tests
 
* Web Services
 
| valign="top" |
 
* Accessibility
 
* Assignment
 
* Assignment (2.2)
 
* Blocks
 
* Blog
 
* Book
 
* Calendar
 
* Chat
 
* Choice
 
* Commenting
 
* Course
 
* Database activity module
 
* Filepicker
 
* Filters
 
* Forms Library
 
* Forum
 
* Global search
 
* Glossary
 
* HTML Editor
 
* HTML and CSS
 
* IMS-CP resource type
 
* Lesson
 
* MNet
 
* My home
 
* Navigation
 
* Ratings
 
* Reports
 
* Resource
 
* Survey
 
* Themes
 
* Usability
 
* Wiki (2.x)
 
* Workshop
 
| valign="top" |
 
* Activity completion
 
* AJAX and JavaScript
 
* Backup: IMS-CC
 
* Badges
 
* Conditional activities
 
* Course completion
 
* Database MS SQL
 
* Documentation
 
* External Tool (IMS-LTI)
 
* Feedback
 
* General
 
* Language
 
* Maths filters
 
* Other
 
* Plagiarism
 
* Policy
 
* Questions
 
* Quiz
 
* SCORM
 
* Security Alert
 
* Survey 2
 
* Translation
 
* Unknown
 
|}
 
  
===Assignee and watchers===
+
=== Revisiting old issues ===
  
Prior to May 2013 issues were [[Changes_to_issue_assignment|automatically assigned]] to developers, even though they might never work on the issue. This created a false sense of progress and notionally denied others from becoming involved. Currently issues are not automatically assigned. Developers should only assign an issue to themselves when they have a definite intention to complete the issue.
+
While searching the tracker you may come over issues that were reported long time ago but still remain open. Again, ask the following questions
  
Some components are automatically watched by users. If you think a user should be involved, add them to the list of watchers and comment on the issue.
+
* '''Is this still an issue?''' If it is not applicable any more either close it or comment about it on the issue and recommend to close. Very few tracker users actually have permission to close issues but the component lead or watchers will see your comment and revisit issue. Another good practice is to replace ''triaged'' label with ''triaging_in_progress'' and add a comment asking if the issue can be closed. If users confirm that it was resolved or nobody replies in 30 days, issue should be closed.
 +
* '''Do affected versions need correction?''' If the issue is still applicable, make sure to add missing current affected versions or comment about it on the issue if you can't edit it.
 +
* '''Does the issue have patch and if yes, is it still applicable?''' If the issue has patch that still works but ''patch'' label is missing, look through comments or history to see if the ''patch'' label was removed after reviewing the current patch. If you find that label was never added, do it yourself. On contrary if the issue has ''patch'' label but the patch is no longer applicable or not sufficient, remove the label.
 +
* '''Are there any duplicating issues?''' When finding duplicates among the old issues it might not be obvious which issue to close as a duplicate. Usually we should leave the first reported issue but if the later issues have more watchers, better description, more votes, useful comments, attached patch, etc. you may decide to close the earlier issue and leave the later. Sometimes both issues have lots of watchers and they both remain open. In any case, always create links between duplicates or related issues.
 +
* '''Does the issue have assignee who forgot about it or misleading status "Development in process"?''' Due to some  [[Changes_to_issue_assignment|process changes]] in May 2013 some issues still have a real user in the ''Assignee'' field but this user actually does not work on the issue. Sometimes ''Assignee'' remains filled after failing peer review, sometimes developers simply forget that the issue was assigned to them. If you suspect that ''Assignee'' does not actually work on the issue, comment asking they about it and in some cases remove the ''Assignee'' completely. Allow somebody else to work on the issue without feeling that the issue is "taken". Please also note that for some time tracker had a restriction that ''Assignee'' could not be empty so you can find lots of issues assigned to ''moodle.com'' or ''Nobody''. Do not modify such issues as this creates unnecessary activity, emails to watchers and irrelevant change in the "Last update date".
  
===Testing instructions===
+
=== Comments templates ===
 +
====Redirecting someone with a support request====
 +
<p class="note">
 +
Thanks for taking the time to create an issue. However, the problem you describe seems to be specific to your site, since I am unable to reproduce it on http://demo.moodle.net/. Thus I suggest you post asking for help in one of our community forums https://moodle.org/community/.<br /><br />
 +
I'm going to close this issue now. If you find that your problem can indeed be reproduced on http://demo.moodle.net/ or you have a suggestion for an improvement, please create a new issue providing as much detail as you can.</p>
  
; Has the reporter provided testing instructions?
+
====Redirecting someone making a 'feature request' that is actually already possible====
: The Testing instructions field is used during testing, after an issue has been resolved. If the reporter has provided appropriate testing instructions they can be left in place and updated by the developer who works on the issue.
+
<p class="note">
: Often issue reporters will include reproduction steps in this field. It is good that this information is provided by the reporter, but it should be moved to the description unless it is seen to constitute an appropriate set of testing instructions.
+
Thank you for taking the time to request a new feature / suggest an improvement. However, what you ask for is already possible. I suggest you post asking for help about how to set up what you want in one of our community forums https://moodle.org/community/.<br /><br />
 +
I'm going to close this issue now.</p>
  
===Labels===
+
====Refer the user to their instance administrators====
 +
<p class="note">You have posted an issue on the Moodle Tracker, which relates to the improvement of Moodle code, not the Moodle site you are using at your institution. I recommend you contact the administrators of your Moodle site.</p>
  
; Are [[Tracker issue labels|standard labels]] labels used?
+
====Closing duplicate====
: If not, remove superfluous labels (if necessary, they can be copied to the description).
+
<p class="note">
:* Issues are labelled ''triaged'' to indicate that all the above points have been covered.
+
Thanks for your report.<br /><br />
:* Issues with proposed fixes are labelled ''patch'' so that they can be found easily and given attention. When this is the case, consider whether moving the issue to the 'Waiting for peer review' state in the workflow might be more appropriate. Also consider if the user should be promoted to a jira-developer so they can submit patches more easily in future.
+
It seems you're not the only one to have come across this bug, as it's been reported previously - see MDL-xyz. I'm going to close this issue now so we can focus on MDL-xyz. Please watch, vote or comment on MDL-xyz if there is any additional information you can provide.
 +
</p>
  
: A non-standard label that can be useful is 'addon-candidate', which can be used to signify that an improvement or new feature could be achieved as an add-on. This way, people looking for projects could search for issues with this label.
+
====Issue affects only unsupported versions====
 +
<p class="note">
 +
Thanks for your report. However, it seems that this problem does not affect current versions of Moodle. Thus, you are recommended to upgrade your installation in order to take advantage of recent fixes and improvements. See https://docs.moodle.org/en/Upgrading for details.</p>
  
===Gratitude and encouragement===
+
====Requesting more information====
 +
<p class="note">
 +
Thanks for reporting this.<br /><br />
 +
Could you please add more information to your report such as replication instructions, error messages and screenshots. If you are able to suggest a workaround, that would be immediately helpful to others experiencing this problem. If you can determine the cause of the problem or even determine a solution, that will be very valuable.
 +
</p>
  
After triaging many issues, it's easy to lose sight of the fact that the reporter has contributed their time and energy to report an issue for the benefit of the community.
+
====Request to correct translation====
 +
<p class="note">
 +
Thanks for reporting this.<br /><br />
 +
<nowiki>The problem that you are describing refers to the translation in another language. Language string problems can be corrected by [Contributing a translation|https://docs.moodle.org/dev/Contributing_a_translation] or by contacting the language pack maintainer as listed in the [Translation credits|http://lang.moodle.org/local/amos/credits.php]. I'm going to close this issue because Moodle does not include translations in the standard download package.</nowiki><br />
 +
</p>
  
It is easy to become defensive of Moodle if reports are seen as criticism (and sometimes reporters may use phrasing that suggests this), however the triagers response must always be one of sincere gratitude.
+
====Correction to English language strings====
 +
<p class="note">
 +
Thanks for reporting this.<br /><br />
 +
Please note that English language string typo fixes and suggested improvements can be <nowiki>[contributed to the English (fixes) (en_fix) language pack|https://docs.moodle.org/dev/Improving_English_language_strings]</nowiki>
 +
</p>
  
It is also important to encourage reporters to continue being involved with the issue after it is triaged. We must not give the sense that we are taking the issue ownership away from the reporter. Instead the reporter should be encouraged to discover the cause of the problem and present a solution; this is appropriate in an open-source project. It is amazing that such a challenge can lead to a sense of purpose for the reporters.
+
====Bug in contributed plugin (plugin can be identified)====
 +
<p class="note">
 +
You have created an issue under MDL project in the tracker. However the problem that you are describing refers to contributed plugin xxx. <br />
 +
<nowiki>I have moved your issue in the appropriate component of the CONTRIB project in tracker. I will also recommend you to leave a comment on the plugin page in the [Moodle plugins directory|https://moodle.org/plugins/]</nowiki>
 +
</p>
  
Here is a comment thanking a user for reporting a '''bug''' and encouraging them to continue working on the issue.
+
====Triaging a bug request====
 
<p class="note">
 
<p class="note">
 
Thanks for reporting this.<br /><br />
 
Thanks for reporting this.<br /><br />
I've put that on the backlog.<br /><br />
+
I've put it on the backlog.<br /><br />
 
In the meantime feel free to help us work on this issue. If you are able to provide a patch or links to your Git repository branch, please add a patch label so we will spot it.
 
In the meantime feel free to help us work on this issue. If you are able to provide a patch or links to your Git repository branch, please add a patch label so we will spot it.
 
</p>
 
</p>
  
Here is a comment thanking a user for suggesting an '''improvement''' and encouraging them to continue working on the issue.
+
====Triaging an improvement or new feature====
 
<p class="note">
 
<p class="note">
 
Thanks for reporting this.<br /><br />
 
Thanks for reporting this.<br /><br />
Line 278: Line 188:
 
*[[Process]]
 
*[[Process]]
 
*[[Talk:Bug triage]]
 
*[[Talk:Bug triage]]
 +
*[[Tracker tips]]
  
 
[[Category:Processes]]
 
[[Category:Processes]]
 
[[Category:Quality Assurance]]
 
[[Category:Quality Assurance]]
 
[[Category:Tracker]]
 
[[Category:Tracker]]

Latest revision as of 11:05, 4 October 2018

alt Triage image

Triage is medical term referring to the process of prioritising patients based on the severity of their condition so as to maximise benefit (help as many as possible) when resources are limited.

Bug triage is a process where tracker issues are screened and prioritised. Triage should help ensure we appropriately manage all reported issues - bugs as well as improvements and feature requests.

Triage initially happens shortly after the issue was reported but it can be repeated later if the initial assumptions were wrong, issue was resolved otherwise, affected versions need updating or there are other reasons to review the issue.

Get involved

Anybody can do triage in form of correcting the components and/or affected versions, linking to related issues, and of course commenting asking for clarification, confirming bug, redirecting to forum, etc. Users in jira-developers and moodle-triage groups can edit any issue, jira-users can comment on any issue or edit issues they reported. Please see MDLSITE-3592 if you are not a developer but would like to help with triage process.

Adding triaged label and placing the issue on the backlog should only be done by component lead or HQ developer. At the same time other users can remove triaged label from the old issues or replace it with triaging_in_progress if they want to request an additional triage.

The triage process

Initial screening

First of all, identify the issues that should be closed or placed under investigation. Ask the following questions:

  • Is the issue a request for support/help? If so, the reporter should be directed to the forums to seek help and the issue should be closed as "Not a bug". Sometimes improvement requests can be phrased as a question, though; if this is the case, ask the reporter to reword the description to describe an improvement.
  • Have the reporter mistaken the Moodle Tracker with their own support desk? Sometimes people mistake the Moodle Tracker as a place to request help about their own Moodle instance, often about logging in. We need to refer the user to their instance administrators and close the issue as "Not a bug".
  • Has the issue been reported previously? If so, link to a duplicate issue and close the newly reported issue as a "Duplicate" with no fix version set. Encourage the reporter to search before reporting. If a newer issue has a patch or more voters/watchers, consider closing the older issue. Checking for duplicates first will save you having to check the rest of the issue. See Tracker tips for help with effective search of tracker.
  • Does the problem affect only unsupported versions? If so, the issue should be closed using "Fixed" (preferred as it sounds better) when the issue is resolved in current versions or "Not a bug" when the issue has disappeared due to changes leading to current versions. See [Releases] to find currently supported versions
  • Did the problem arise because of mistake in documentation or lack thereof? If it appears that the reporter does not understand a particular feature in Moodle and the documentation is lacking, ask the reporter where would he expect to find documentation about it. Then simply edit the relevant pages in the documentation wiki and close the issue. If required change is significant, add Documentation component and docs_required label.
  • Is the problem a language string change? Language string problems can be corrected by contributing a translation or by contacting the language pack maintainer as listed in the Translation credits. English language string typo fixes and suggested improvements can be contributed to the English (fixes) (en_fix) language pack or given the component 'Language' for fixing by the Language component lead. Such issues should be closed in the Tracker using "Deferred".
  • Is it a usability issue? If so, add the component "Usability" in addition to the component(s) specifying the area of Moodle.
  • Was the problem caused by additional code or 3rd party plugins? If you can identify the plugin, move the issue to the respective component of CONTRIB project. Otherwise comment and close as "Not a bug".
  • Can this be implemented as a plugin? And maybe the plugin already exists in the plugins directory. Explanation should be given to the reporter that Moodle provides the framework but does not work on any possible plugin. Add label "addon_candidate" but do not close the issue. This can also apply to the requests to significantly redesign existing plugins and it would be more preferable to create a new alternative plugin.
  • Does the problem seem rational? If not, then the problem may simply be an misunderstanding on the part of the reporter. It might be a problem exclusive to the reporter's server set-up. If you can replicate the problem quickly, do so. If you can't replicate the problem, ask the reporter to attempt to replicate the problem on http://demo.moodle.net. If the problem seems persistent but strange, consider asking a developer with experience working in the area to consider the problem and determine if it could be a real problem.
  • Can the problem be replicated? If not, or information on the issue is insufficient, ask the reporter to add error messages, screenshots, environment information (OS, web server, browser) and exact replication instructions

As a result of initial screening up to 20% of new issues may be closed. When closing the issues make sure to set the correct resolution and write a polite comment with explanation, refer to the templates below. If you have doubts, ask the questions and always add label triaging_in_progress. Subscribe to the filter My old issues in triage and you will receive notifications after 30 days of inactivity on such issues. See Tracker tips about how to subscribe to filter. Very often the reporter never follows up on their own issues and this is a good way to find such issues and ping reporter again or make a final decision about closing.

Confirming the issue

When you confirm that issue is indeed a bug or a reasonable improvement request that was not reported previously, make sure that the following is accurate:

  • Security level. Security level must be set as soon as possible if the reported bug discloses vulnerability in Moodle that can be exploited to access information without appropriate level, create an attack on the site, embed XSS or forge a request. In some rare cases Improvements may be also marked as security
  • Summary. Summary of the issue should clearly describe the problem or improvement area, rephrase such summaries as "Some improvements in xxx" or "Error in Moodle", etc.
  • Issue Type. The following issue types are used in Moodle:
    • Bug - represents an actual bug and should be fixed in all supported versions. Very often when reporter expects something to be better than it actually is they call it a bug when it's in fact an improvement.
    • Improvement - improvement to existing functionality. If addressed, will be integrated in the following major release only
    • New feature - completely new feature, also will not be applied to the released versions (unless implemented as a plugin and submitted to plugins directory)
    • Task - usually created by developers themselves and can not be classified as Bug or Improvement, for example, adding automated tests, improving documentation, etc.
    • Epic - created by HQ developers or component leads to collect together issues that represents parts of one project. META issues and sub-tasks should not be used any more
  • Priority. Some reporters over-state an issue's priority. Some reporters don't know they can set a priority. Priority is used as one of the criteria when sorting issues in the backlog, so it should reflect the position of this issue comparing to the others. Usually Improvements have Minor or Major priority and Bugs can have any priority up to Blocker. Priority levels have specific criteria. Please see the Tracker guide
  • Component/s. Listing components correctly is important as they are the primary variable used for searching for issues, also adding a component automatically include the component lead as a watcher. Issue may have several components when needed
  • Affects version. This field should include one or more released *and supported* versions of Moodle that are affected by the issue, with the following exceptions:
    • The issue is a bug in code that is present in the Master branch only, in which case the next major version should be used. (The next major version should not be used in conjunction with previous released versions, this won't make sense later.)
    • The issue is a new feature and is unrelated to any existing code in Moodle, in which case the 'Future dev' version should be used.
  • Labels.
    • Remove functionality tags that some reporters add as labels, only standard labels or partner-specific labels are used in Moodle
    • Issues with proposed fixes are labelled patch so that they can be found easily and given attention. When this is the case, consider whether moving the issue to the 'Waiting for peer review' state in the workflow might be more appropriate
    • Add addon_candidate label if the functionality can be implemented as a plugin;
    • If you are component lead or HQ developer you may also add triaged label to indicate that triage process is completed. Use it only when the issue has actually been added to the backlog

Note that "Fix version" is no longer used during triage. If you are in "moodle-triage" group, you can use the Triage screen to edit only aforementioned fields, see MDLSITE-3592.

When commenting on the issue give more details on replication, environment or testing. Ask questions, add watchers, modify description if needed. Link to as many related issues as possible. Do everything that will make the issues scope more clear and attract opinions, discussions and patches.

Don't forget to show Gratitude and encouragement. After triaging many issues, it's easy to lose sight of the fact that the reporter has contributed their time and energy to report an issue for the benefit of the community.

It is easy to become defensive of Moodle if reports are seen as criticism (and sometimes reporters may use phrasing that suggests this), however the triagers response must always be one of sincere gratitude.

It is also important to encourage reporters to continue being involved with the issue after it is triaged. We must not give the sense that we are taking the issue ownership away from the reporter. Instead the reporter should be encouraged to discover the cause of the problem and present a solution; this is appropriate in an open-source project. It is amazing that such a challenge can lead to a sense of purpose for the reporters.

Following up on issues

Tracker is set up by default so as soon as you comment on any issue or edit it, you become an automatic watcher and any following change on the issue will be emailed to you.

If you have encouraged the reporter well, they may submit a patch or somebody else may do it. Make sure that the patch label is added or issue is sent to peer review. On the contrary, if patch label was added by somebody else but you clearly see that patch is far from ready, remove the label and leave a comment explaining it.

Users may also comment with additional details, screenshots, replicating instructions. It may happen that the issue gets re-evaluated and priority or summary changed.

Revisiting old issues

While searching the tracker you may come over issues that were reported long time ago but still remain open. Again, ask the following questions

  • Is this still an issue? If it is not applicable any more either close it or comment about it on the issue and recommend to close. Very few tracker users actually have permission to close issues but the component lead or watchers will see your comment and revisit issue. Another good practice is to replace triaged label with triaging_in_progress and add a comment asking if the issue can be closed. If users confirm that it was resolved or nobody replies in 30 days, issue should be closed.
  • Do affected versions need correction? If the issue is still applicable, make sure to add missing current affected versions or comment about it on the issue if you can't edit it.
  • Does the issue have patch and if yes, is it still applicable? If the issue has patch that still works but patch label is missing, look through comments or history to see if the patch label was removed after reviewing the current patch. If you find that label was never added, do it yourself. On contrary if the issue has patch label but the patch is no longer applicable or not sufficient, remove the label.
  • Are there any duplicating issues? When finding duplicates among the old issues it might not be obvious which issue to close as a duplicate. Usually we should leave the first reported issue but if the later issues have more watchers, better description, more votes, useful comments, attached patch, etc. you may decide to close the earlier issue and leave the later. Sometimes both issues have lots of watchers and they both remain open. In any case, always create links between duplicates or related issues.
  • Does the issue have assignee who forgot about it or misleading status "Development in process"? Due to some process changes in May 2013 some issues still have a real user in the Assignee field but this user actually does not work on the issue. Sometimes Assignee remains filled after failing peer review, sometimes developers simply forget that the issue was assigned to them. If you suspect that Assignee does not actually work on the issue, comment asking they about it and in some cases remove the Assignee completely. Allow somebody else to work on the issue without feeling that the issue is "taken". Please also note that for some time tracker had a restriction that Assignee could not be empty so you can find lots of issues assigned to moodle.com or Nobody. Do not modify such issues as this creates unnecessary activity, emails to watchers and irrelevant change in the "Last update date".

Comments templates

Redirecting someone with a support request

Thanks for taking the time to create an issue. However, the problem you describe seems to be specific to your site, since I am unable to reproduce it on http://demo.moodle.net/. Thus I suggest you post asking for help in one of our community forums https://moodle.org/community/.

I'm going to close this issue now. If you find that your problem can indeed be reproduced on http://demo.moodle.net/ or you have a suggestion for an improvement, please create a new issue providing as much detail as you can.

Redirecting someone making a 'feature request' that is actually already possible

Thank you for taking the time to request a new feature / suggest an improvement. However, what you ask for is already possible. I suggest you post asking for help about how to set up what you want in one of our community forums https://moodle.org/community/.

I'm going to close this issue now.

Refer the user to their instance administrators

You have posted an issue on the Moodle Tracker, which relates to the improvement of Moodle code, not the Moodle site you are using at your institution. I recommend you contact the administrators of your Moodle site.

Closing duplicate

Thanks for your report.

It seems you're not the only one to have come across this bug, as it's been reported previously - see MDL-xyz. I'm going to close this issue now so we can focus on MDL-xyz. Please watch, vote or comment on MDL-xyz if there is any additional information you can provide.

Issue affects only unsupported versions

Thanks for your report. However, it seems that this problem does not affect current versions of Moodle. Thus, you are recommended to upgrade your installation in order to take advantage of recent fixes and improvements. See https://docs.moodle.org/en/Upgrading for details.

Requesting more information

Thanks for reporting this.

Could you please add more information to your report such as replication instructions, error messages and screenshots. If you are able to suggest a workaround, that would be immediately helpful to others experiencing this problem. If you can determine the cause of the problem or even determine a solution, that will be very valuable.

Request to correct translation

Thanks for reporting this.

The problem that you are describing refers to the translation in another language. Language string problems can be corrected by [Contributing a translation|https://docs.moodle.org/dev/Contributing_a_translation] or by contacting the language pack maintainer as listed in the [Translation credits|http://lang.moodle.org/local/amos/credits.php]. I'm going to close this issue because Moodle does not include translations in the standard download package.

Correction to English language strings

Thanks for reporting this.

Please note that English language string typo fixes and suggested improvements can be [contributed to the English (fixes) (en_fix) language pack|https://docs.moodle.org/dev/Improving_English_language_strings]

Bug in contributed plugin (plugin can be identified)

You have created an issue under MDL project in the tracker. However the problem that you are describing refers to contributed plugin xxx.
I have moved your issue in the appropriate component of the CONTRIB project in tracker. I will also recommend you to leave a comment on the plugin page in the [Moodle plugins directory|https://moodle.org/plugins/]

Triaging a bug request

Thanks for reporting this.

I've put it on the backlog.

In the meantime feel free to help us work on this issue. If you are able to provide a patch or links to your Git repository branch, please add a patch label so we will spot it.

Triaging an improvement or new feature

Thanks for reporting this.

Adding more detail to your suggestion will make it easier to work on.

If you can propose a code solution, that will help others who may have the same need and will increase the chance of this improvement/feature coming about sooner. If you are able to provide a patch or links to your Git repository branch, please add a patch label so we will spot it. Otherwise, how about posting in a forum on moodle.org and encouraging people to vote, comment and/or come up with a patch.

Creating a filter and gadget for triaging

If you are a component lead you are responsible for triaging issues that involve your component. A good way to monitor newly created issues is to create a filter in the Tracker and add a gadget on your dashboard to show the results of the filter.

Adding a filter

In Tracker...

  1. Select Issues > Search for Issues.
  2. Create a search for untriaged issues in your component, for example...
    component in (Assignment) AND resolution = Unresolved AND (labels is EMPTY OR labels not in (triaged)) ORDER BY created DESC
  3. Run the search query by pressing Enter or clicking the magnifying glass icon to the right of the search box.
  4. When the search query results are displayed, click the Save as button.
  5. Save the search query with an appropriate name, like "Untriaged Assignment issues".

Adding a filter in a gadget to your dashboard

In Tracker...

  1. Click Dashboards > Dashboard for XXX (where XXX is your name).
  2. Click + Add Gadget.
  3. Find the Filter Results gadget.
  4. Click Add it Now.
  5. Click Finished.
  6. In the Saved Filter input, search for and select your newly created filter.
  7. Click Save.

The most recent untriaged issues should appear in reverse-date order.

Triaging priorities and the Triaging Dashboard

The following are the priorities for ordering issues to be triaged that are reflected on the Triaging dashboard.

  1. Security issues - should always be reduced to 0
  2. High-priority issue - aim for blockers and critical issues to be reduced to 0
  3. Partner issues - aim for partner issues to be reduced to 0
  4. Patched issues - aim to triage as soon as possible
  5. Developer-reported issues (HQ and non-HQ) - should be quick to triage
  6. Recent community bugs - should be triaged last-in-first-out

See also