Peer reviewing

Process

Peer review process helps to prepare the issue for integration. The peer reviewer is another developer who was not involved in the development process on the issue and therefore can take a fresh look and notice something that original developer might have forgotten during development. It is important to check that the bug actually is present and the code fixes it without creating new regressions.

Completing Peer review as a community member

Most developers external to Moodle HQ do not have the ability to complete an official "Peer review" by progressing the JIRA statuses. This should not discourage you from looking at other contributors code and providing comments and feedback. This allows for much quicker reviews by the final reviewer who can conduct the official Peer review for progression to integration. You can consider that official review as a check and verification that your review contribution has covered all the issues. If that's the case they can progress the tracker item very quickly without significant work.

To provide feedback to the developer, leave the issue in "Waiting for Peer Review" as that is the official review status. Proceed to review the code based on the guidelines below, including any appropriate comments. Once finished, please post a comment clearly stating the outcome of your review. If you think it needs more work then indicate what needs to be changes. If you are happy with the patch, leave a clear comment that you believe this is ready to be made part of Moodle. This can then easily be seen by a HQ developer or component lead and they can quickly take appropriate action.

If a followup review happens to identify something you did not find, you have an opportunity to expand your knowledge and provide better reviews in the future as well as having saved everybody else some time.

Your review may not progress the issue in the tracker, however it will increase the ability for any issues reviewed to be included sooner as the code will be of much higher quality as it progresses through the development process.

Feedback to indicate the issue is ready to progress might look like the following;

I have reviewed your patch, it addresses the problem and complies with Moodle standards and I believe it's ready to progress further.
There may be further reviews before this issue progresses to integration. Those reviewers be have additional questions at those stages. Please keep following this tracker issue as others review it.
Thanks again for your contribution.

Feedback to indicate the issue requires further work might look like the following;

Thanks for providing a patch.
Unfortunately this patch does not fully address the reported issue. ... DETAILS...

Even though the code does solve the issue in the short-term it is very likely to create regressions .....

I have spent some time reviewing the patch and I would recommend that you .....

Please indicate If you are willing to continue working on this issue and complete the solution.

Peer review for development by HQ or a known common contributor

When code comes from a HQ developer or external developer who has been contributing significantly to Moodle and is well acquainted with Moodle standards, peer review is limited to checking the code according to the Checklist below.

If everything is fine, the peer reviewer submits the issue for integration.

If some additional work is needed or the author specifically asked not to submit for integration yet, the peer reviewer clicks on “Finish peer review” and the issue state returns to “Development in progress”. Usually the name of the peer reviewer stays on the issue and if a second peer review is requested it is expected that the same Peer reviewer picks it up. If the peer reviewer is not able to do the second review, they should remove their name and comment about it. Otherwise the issue does not appear on “waiting for peer review” dashboard. Please remember that not all jira users have permission to submit for integration.

Peer review for external developers

When the code has come from an external developer, the peer reviewer will also help the developer to lead the issue to integration. In this case the peer reviewer should not use “Finish peer review” button.

If the issue needs additional work, the peer reviewer comments about the suggested changes but does not change the status of the issue and it remains as “Peer review in progress”. If the author of the patch does not reply in 4 days, the peer reviewer may select either to complete the patch themselves or reopen the issue. The decision should be based on the amount of work required to complete the solution, for example, changing coding style or commit message, rebasing, backporting, adding testing instructions, etc. should be performed by the peer reviewer. This may require picking the commits into reviewer’s git branch.

It is very important to give the credit to the author of the code by either keeping their authorship on the commit or adding “Thanks to XXXX for providing the patch” to the commit message. If the author of the patch is not in jira-developers group the special user 'moodle.com' should be entered in Assignee field.

There could be situations when the patch is incomplete, does not fix the original issue, creates regressions or otherwise is not correct. As stated above, the peer reviewer should comment about it and wait for the feedback from the author. If there is no feedback, or the author can not work on this issue any more, and the peer reviewer also does not find it easy and important enough to complete, the issue needs to be reopened. Button “Fail peer review” (available to HQ developers only) will reset the issue status to “Reopened”. On this screen peer reviewer should remove assignee, peer reviewer and “patch” label from the issue. This way issue will become available again for anybody who want to work on it. All communication will remain in comments and issue history.

If the issue has passed peer review but the integrator or tester has raised some questions about it, then normally the developer who created the patch would be expected to respond. If they do no respond quickly enough, then the peer reviewer is expected to step in and take responsibility for the change they reviewed.

Once the issue is ready for integration, you can submit it to integration on behalf of the developer. Most external developers (those who are not in the jira-developers group) do not have permission to submit their own issues to integration so cannot do it themselves.

Replies templates

Thanks for providing a patch.
I have reviewed your code and can confirm that it addresses the reported issue. We would like to include it in core. Moodle values its contributors and tries to give them credit when possible. If you are interested in your name appearing on the https://moodle.org/dev/contributions.php page you can create a git commit that we will then pull into Moodle. You can learn more about Git and how Moodle uses it at [Git for developers|https://docs.moodle.org/dev/Git_for_developers] page. Please let me know if you want to prepare a git branch. Or if you don’t have time to go through the whole process at the moment I can pick your patch myself.

Thanks for providing a patch.
Your code looks almost ready for integration into Moodle. There are just some little things that need to be changed to comply with Moodle standards. Can you please take a look at the review results below and tell me if you are able to modify your branch to address them.

I have reviewed your patch, it addresses the problem and complies with Moodle standards. I'm pushing this issue for integration. Following Moodle [Process|https://docs.moodle.org/dev/Process] it will go through integration review and testing before being included in the product. There might be additional questions from an integrator and/or a tester at those stages. It would be appreciated if you read tracker emails and can reply to questions if needed. If everything goes well during the next two stages your issue will be included in the next weekly release and your count on https://moodle.org/dev/contributions.php page will increase.
Thanks again for your contribution.

Thanks for providing a patch.
Unfortunately this patch does not fully address the reported issue. ... DETAILS...

Even though the code does solve the issue in the short-term it is very likely to create regressions .....

I have spent some time reviewing the patch and I would recommend that you .....

Please let me know If you are willing to continue working on this issue and complete the solution.

Checklist

These are points to consider while peer-reviewing issues. Further explanation below.

[] Syntax
[] Whitespace
[] Output
[] Language
[] Databases
[] Testing (instructions and automated tests)
[] Security
[] Documentation
[] Git
[] Third party code
[] Sanity check
[] Icons

Acceptable check-marks are Y (for yes), N (for no) or - (for not applicable). All N check-marks should be accompanied by an explanation of the problem that still needs to be addressed.

Syntax

To allow the community of Moodle developers to work together, conventions should be followed.

• The code is easy to understand and, where it isn't, comments have been provided.
• Variables are named correctly (all lower case, no camel-case, no underscores).
• Functions are named correctly (all lower case, no camel-case, underscores allowed).
• PHP DocBlocks have been updated and adhere to coding style guide.
• Where functions are being removed, the Deprecation process is followed.
• The code doesn't use deprecated functions.
• $_GET, $_POST, $_REQUEST, $_COOKIE, and $_SESSION are never used.

See the Coding style guide for details.

Whitespace

Unnecessary whitespace changes can cause conflicts when committing code.

Ensure that:

• there are no unnecessary blank lines in the new code;
• blank lines do not contain spaces;
• there are no unnecessary changes to whitespace in other areas on the file;

Output

Output needs to be controlled by renderers to achieve consistency and correct application of themes.

Ensure that:

• output renders are used to generate output strings, including HTML tags;
• HTML output is valid XHTML;
• no inline styles have been used in HTML output (everything has to be in CSS);
• CSS has been added to the appropriate CSS files (base, specific area, sometimes canvas); and
• the code doesn't use buffered output unless absolutely necessary.
• all visual output has a RTL alternative included

feedback any notices (E_STRICT, etc) seen into the MDL.

Language

To achieve appropriate internationalisation of Moodle, language strings must be managed correctly.

Ensure that:

• new language strings are named correctly (all lower case, no camel-case, underscores are permissible in some cases);
• help strings are named and formatted as described in Help strings;
• language strings are used instead of hard-coded strings for text output;
• language strings have not been removed or renamed, had their meaning changed or had their parameters changed in stable branches (permitted only in master); and
• AMOS commands have been specified when moving or copying language strings.

Databases

DB calls are the greatest performance bottleneck in Moodle.

If there is SQL code you can test quickly then do so.

Ensure that:

• there are minimal DB calls (no excessive use of the DB); and
• the code uses SQL compatible with all the supported DB engines (check all selected fields appear in an 'order by' clause).

Testing instructions and automated tests

It is the developer's responsibility to test code before integration. Issues should not be sent for peer review without tests so that the peer reviewer can assess their quality and use them to consider the scope of the issue.

Ensure that:

• there are specific testing instructions that state how, as well as what, to test. Please ensure that the testing instructions are in the correct format: Behat gherkin;
• new unit tests have been added when there is a change in functionality; and
• unit tests pass for related areas where changes have been made.
• Behat tests pass for related areas where changes have been made, especially when it involved UI changes.

Security

The user community relies on Moodle being responsibly secure.

Ensure that:

• user login is checked where an identity is needed;
• sesskey values are checked before all write actions where appropriate (some read actions as well);
• capabilities are checked where roles differ; and
• if the issue itself is a security issue, the security process is being followed.
  • Ensure that the fix is not available in a public repository (ie. a personal Github account); stand-alone patches should be provided instead.
  • The issue will not be integrated until just before the next minor version release.

Documentation

Work does not stop when code is integrated.

Ensure that:

• Appropriate labels have been added when there has been a function change, particularly
  • docs_required (any functional change, usually paired with ui_change),
  • dev_docs_required (any change to APIs, usually paired with api_change),
  • ui_change (any functional change, usually paired with docs_required, except ui_change remains permanetly),
  • api_change (any change to APIs that devs will need to know about, usually paired with dev_docs_required, except api_change remains permanetly),
  • unit_test_required and acceptance_test_required, when there are api or ui changes needing improved coverage, and
  • qa_test_required (significant functional change, not covered by unit/acceptance ones).
• Also, verify that the components for the issue are correctly set, so maintainers (subscribed by default) will be mailed about issues early in the process.

Git

Ensure that:

• the commit matches the Coding style
• the Git history is clean and the work has been rebased to logical commits; and
• the original author of the work provided as a patch has been given credit within the commit (as author of in the commit message if changes were made).

Third party code

Does the change contain third party code? If so, ensure that:

• The code is licensed under a GPL compatible license.
• The instructions for upgrading/importing the library and contained within a readme_moodle.txt file.
• The library is recorded in a thirdparty.xml file, including licensing information.
• Third party code has been scanned to check for url accessible entry points that could be exploited. These should either be disabled, or if required functionality they should be checked for security weaknesses.
• Does not duplicate the functionality of any existing api or third party library in core.
• Any modifications to third party code are recorded in readme_moodle.txt

Sanity check

Ensure that:

• the code seems to solve the described problem completely within its reported scope (and further issues have been created to resolve remaining parts or further refactoring);
• the code makes sense in relation to the broader codebase (look at the whole function, not just the altered code); and
• the developer has searched for and fixed other areas that may also have been affected by the same problem.
• if any version numbers have been changed in version.php files, then the changes follow the rule for updating version numbers in core.

Icons

Are new icons being introduced? If so, ensure that:

• the icons abide by our icon guidelines with regards to size, design and format
• the icons are do not unnecessarily add new ways of expressing existing concepts
• the icons are in a pix folder that makes sense

See Also

• Code checker plugin