Talk:Bug triage

Jump to: navigation, search

Requesting input from other developers

Sometimes it's hard to know when somebody has been added as watcher in order to require input from him/her. That's because own watches aren't different at all from "request watches" from other developers.

IMO it could be really interesting to have one new field, possibly only for developers and testers, in order to request input from other moodlers. That way, we could have one simple filter, call it, "Bugs I've been requested to input about". By taking a quick look to that list daily I think we could improve the decision process in a nice way.

Commented with Martin and agreed about to suggest it here. Thanks! --Eloy Lafuente (stronk7) 10:51, 31 March 2008 (CDT)

Eloy, I like your idea, however I don't know a way of creating a field for developers and testers only. (I wish fields like QA assignee and difficulty could be removed from the create issue form to make the form shorter.) Perhaps we could implement your idea by reassigning the issue to someone when you want their input and adding a comment like 'Reassigning to Helen for her input on whether more documentation is needed. Please reassign back to me afterwards.' --Helen Foster 13:02, 3 September 2009 (UTC)
Yes, temporal reassigning could be a solution, but main problem is that the original assignee can lost the track of the issue easily (if forgets to add him/her-self as watcher or so). Also, that won't solve the problem of searching for those bugs (based in one "free-content" string, any typo or change in the entered text will cause the issue to not appear in the list of "Requesting my opinion". Those are the reason about to have one specialised field for easier searching. About the fields... I haven't tested that... but... aren't the QA and difficulty fields only available (for input) to some groups within Jira? Do a simple Jira user have those fields available when creating new issues? I hope no. --Eloy Lafuente (stronk7) 13:58, 3 September 2009 (UTC)
Edited: Crap! I just created a new user in Jira and, when creating new issues, the QA and difficulty fields are available :-( --Eloy Lafuente (stronk7) 13:58, 3 September 2009 (UTC)
Edited 2: Seems to be an old JIRA limit: http://jira.atlassian.com/browse/JRA-4616 (hackable, see last comment, but I wouldn't recommend it). --Eloy Lafuente (stronk7) 14:13, 3 September 2009 (UTC)
Yes, the only field which is not available to an ordinary Jira user is the assignee field. --Helen Foster 14:09, 3 September 2009 (UTC)
I'm not sure we need this. I think there is a risk of over-engineering the bug database when all you need are comment. If you want input from a particular person, add them as a watcher, then add a comment with the question you want them to answer. Sometimes we need extra fields like QA assignee, but this is not one of them, I think.--Tim Hunt 19:41, 13 September 2009 (UTC)

Sharing the responsibility of bug triaging

I've been rewording Bug triage to hopefully make the two-step process a bit clearer. I was thinking that I could help with step 1, as could any developers new to Moodle. Senior developers could then help with step 2.

Commenting promptly on new issues will act as an encouragement to bug reporters. Not having to check whether a bug can be reproduced should help senior developers triage bugs more efficiently.

Any comments on this way of sharing the responsibility of bug triaging? --Helen Foster 14:23, 3 September 2009 (UTC)

Life after triage

Bugs are removed from the triage list by giving them a fix version or closing them. What happens to all unresolved bugs then? How can we keep the list of triaged bugs as low as possible? Here are some suggestions from Eloy:

  • Encourage all Moodle HQ developers to identify and fix a minimum of 3 stable bugs per week. Perhaps send an email reminder.
  • 'Perhaps also we could introduce, one day per week, to have a quick review in the developer chat about some bugs needing decision. So, while we are triaging... we comment with some "keyword" and then can look for those or so.'

--Helen Foster 14:42, 3 September 2009 (UTC)

BUG: Bugs are not removed from the triage list

"Bugs are removed from the triage list by giving them a fix version..." - does not seem to workBug_triage

For example, MDL-20270 still appears at http://tracker.moodle.org/secure/IssueNavigator.jspa?mode=hide&requestId=10231 even I set a fix version to it

--David Mudrak 08:21, 14 September 2009 (UTC)

David, thanks for your observation. I've just edited the Recent unresolved bugs that need triage filter so that issues with fix versions are now omitted. I also checked that the fix version field doesn't appear in the create issue form for non-developers. Developers (i.e. users in the jira developers group) can of course create new issues with fix versions and so bypass the triage process. --Helen Foster 13:05, 14 September 2009 (UTC)

More triage brainstorming

To agree (and impl) ASAP

  1. Only triagers, issue assignee and, exceptionally, admins (that should be reduced to the minimum), should have permissions to resolve issues (hence define the fixfor version). GH sprints (scrum masters) can be assigned by the schedule issues permission instead.
  2. What happens with all the improvements/new features stuff. All them go to DEV backlog by default? That will cause us having such backlog fill with thousands of issues (a lot of them being crazy ideas, never being to be implemented things...). How should those issues be triaged? Note it is a problem later for scrum managers/product managers when picking issues and o on (not for the triaging process itself). Also, any current modification in the fixfor version should be done with explanation about it or ppl can become really annoyed seeing bugs being moved from, say, 2.0.1 to "none" or so. Or also, "silent" (without notification) moves can be done.... to decide! For example, how should one thing like MDL-18093 triaged?
  3. Sometimes (apart from errors that will happen for sure) the triaging team won't be able to decide if something must be sent to backlogs or merely ignored, thanked and closed as won't be implemented. That's something to address by the PM (talking about improvements/new features). How should the request to the PM be articulated? (online hi!, weekly report, some label like "pmrequested"...).

Ideas from Eloy:

Use the new labels in the Tracker, to tag already triaged bugs with "triaged" or so, that will help looking both for non-triaged bugs and to see "how-many" we have triaged x unit of time (week, month... whatever). If you are in the middle of a triaging session, you add the label. If not, and it's one bug created and assigned in that moment, we don't use the label. That way, each week the triaging team will be able to say: this week xxx bugs triaged. Also, we can use "re-triage" in case somebody request to change triaging (though I don't think it's usual). Note triage can be, "trg" or whatever acronym we use, can be: p1 (phase1=triaging), others can use p2, p3.... so, at any moment it will be really easy to know in which situation one bug is.

Consider a JIRA add-on for detecting duplicates (ideally, when the user introduces it) e.g. http://www.suggestimate.com/ (automated duplicates on creation for JIRA, free for open source projects)

Have an automated system able to:

  1. Get any nobody/moodle.com not-triaged bug.
  2. Calculate the better assignee (from HQ) with some % of confidence in the calculation (based not only in component but in other things like number of bugs, number of bugs fixed in the same component)
  3. If the % of confidence > than, say, 80%, automatically do the assignment.
  4. Else, allow us to manually pick the best assignee.

It is not triage, but moving from nobody/moodle.com to best assignee. For us it would be simply one page with one list and some checkboxes to validate the results of the automatic assignment.

We need to decide what to set in the triaging process: only fix for or also always assignee (because that is another role "product backlog/master" in MD explanations), in other words specify clearly what has to be done in each phase of the issue:

  • triaging
  • backlog mashup
  • scrum/whatever they are going to use
  • development
  • test
  • development review
  • QA
  • land to codebase

or we'll end mixing everything, with landing happening before QA, triaging after development... and all those sort of nasty things happening now.

Another idea - error reporting:

  • Imagine Moodle shows one error/exception
  • Each time that happens, internally Moodle calculates one hash of the trace of the error and shows it. That hash is uniquely based in the information provided by the trace (those lines being showed with debugging enabled saying error in line xxx of yyyy and so on). Obviously we should take out some varying information from the hash (like line numbers or complete paths).
  • In the tracker we allow those hashes to be introduced, so later they can be used both for finding already fixed problems and duplicates.

More yet, sites allowed to do so could be publishing those errors straight to Moodle Tracker, getting 1-vote for each site that has reported the problem, I think that can help a lot with error-based issues, and also, from sites allowed to do so... we would be getting INSTANT reports about real errors.

Idea from Andrew:

Either as a one off manual process or as an ongoing automatic process we could auto-close old issues:

  • Issues that have been resolved for 12 months.
  • Issues that are open but which have been inactive for 2 years and have less than 3 votes.

Looking to get issues closed that are either dealt with or are likely so old that they are irrelevant or are actually duplicates of issues that have been resolved.


Refinement by Tim: (Warning, over-engineering alert!)
A more sophisticated proposal would be a script that automatically adds a comment and a label to bugs that have not been touched in ages, saying that if htere is no meaninful activity for a month ,then the bug will be closed.
Meaningful activity is defined as someone with suitable permissions removing the 'toberemoved' label. Not just someone adding a comment that is not sufficient to persuade a triage-er to remove the label.
Then a second auto script to close labeled bugs after a month.
For example, there are some good, but very old quiz feature requests that have not been touched in years. It would be better to keep them open.
Andrew, Tim: Yeah, a bunch of automatisms will be introduced (carefully and after agreement). Problem is that, right now, Helen and me are starting to triage (manually) the BIGBAG of pending issues (6800), filling the stable/dev backlogs while trying to handle all the new daily issues requiring attention. I think that once we have those manual processes under control, we'll be able to start thinking into automatisms like the exposed above. Of course feel free to add more and more ideas... They will be considered/discussed/agreeed soon. Thanks! Eloy Lafuente (stronk7) 12:02, 8 December 2010 (UTC)

Re. backup issues being triaged to stable backlog

Explanation from chat with Eloy:

About "backup issues being triaged to stable backlog", the idea beyond that decision is that, while Moodle 2.1 (DEV) starts... any change in one module/.... means that backup & restore also needs to be modified in DEV... but those changes will be assigned to the STABLE team, as a way to keep them with the modifications done in DEV. It's a measure to try that the STABLE team is "in touch" with modifications in DEV. Else, when DEV (2.1) is released... the STABLE team won't know much about it. And as far as backup & restore is 100% transversal (covers the whole Moodle), making the STABLE team the one responsible for backup & restore we keep them connected to DEV development. That's the reason for having any backup & restore bug always assigned to STABLE team. To keep them connected to the DEV work/changes.

--Helen Foster 11:36, 31 January 2011 (UTC)

See also

Proposal: LATER target milestone

I think there is a useful difference between:

DEV backlog 
would be nice to fix this in 2.1 if we had the time. If not, consider it for future releases; and
 ???? 
no real intention to fix this. It is not a ridiculous idea, but there is little prospect of it getting done soon.

You might use that second option for a feature request that does not really fit into that part Moodle as it is now. Next time major changes to that part of Moodle are planned, this idea should be considered again, to see if it could be accommodated. Until then, it is not worth considering every time we review the DEV backlog.

I suggest we create a new target milestone LATER to cater for the second case. If this had existed, I would have used it for MDL-27015--Tim Hunt 14:41, 29 March 2011 (UTC)

I don't like the use of Fixed for old bugs

I think that Fixed should be reserved for when we actively fix the bug by making a change to the code. When a new but is reported that was already fixed, well, technically that is a duplicate, if you can be bothered to find the original. Otherwise, I would prefer cannot reproduce for things that were fixed in the past.--Tim Hunt (talk) 15:02, 27 May 2014 (WST)