Difference between revisions of "Release process"
(→4 weeks prior: putting QA tests on hold)
(Add link (only meaningful when within continuous))
|Line 252:||Line 252:|
| style="text-align:center" | ✓
| style="text-align:center" | ✓
| Review the list of features and improvements submitted before code freeze which are awaiting integration review and decide on related QA tests to be put on hold.
| Review the list of features and improvements submitted before code freezewhich are awaiting integration review and decide on related QA tests to be put on hold.
| Development Manager
| Development Manager
Revision as of 08:45, 19 May 2020
8 weeks prior
|1.||✓||Recurring release epics review:||Integration Team|
|2.||✓||Ensure we are running latest/good behat, phpunit and nodejs (npm) versions.||Integration Team|
|3.||✓||Confirm external testers daily availability from -6w to release (internally with dev manager and then, externally, with contractors).||Integration Team|
|4.||✓|| - Move QA tests for new features in old release with automated tests from MDLQA-1 to MDLQA-5249.
- Remind devs:
- Contact with external testing towards preparing the continuous integration period (-6w to +2w, right now) starting in just 2 weeks.
|5.||✓|| - Create a "Must fix for X.Y" version in the Tracker and adjust the "TR - Set integration priority to one" and "TR - Move reopened out from current" jobs in all CI servers to point to it.
- Apply the following changes to filters and dashboards:
7 weeks prior
|1.||✓||Full demo of new code and sign-off for internal HQ projects. Decide which projects will be completed by the code freeze.||Lead Developer|
|2.||✓||Check closed qa_test_required-labelled issues and create new QA tests as required.||Testing Maintainer|
6 weeks prior
|1.||✓|| The Continuous Integration period begins. Warn about it everywhere (telegram, exposed posts...).
Rolling (on demand, beta, rc... all them together with stable weeklies) happens often (Tuesday & Friday are the usual days). All the team is on integration 100% since this week and until the end of continuous.
|2.||✓||This week, with the new schedule, the normal integration cutoff doesn't make any sense and issues will be kept under current integration, instead of being moved / delayed, so the following jobs have to be disabled:||Integration Lead|
|3.||✓||Create the next "X.Y+1" (next dev) version in the Tracker (MDL and CONTRIB), so people can move delayed stuff to next major release if needed.||Integration Lead|
|4.||✓||Warn external developers about the impending code freeze in a post to the General developer forum. (example)||Development Manager|
5 weeks prior
|1.||✓||Confirm the code freeze by replying to the previous warning in the General developer forum.||Development manager|
|2.||✓|| This week, once the integration server has ended with Monday's job moving issues to integration (after 11:00 UTC), the following jobs have to be disabled:
Issues will be manually picked for integration until the post-release "On Sync" period ends, usually 2 weeks after release.
Review all the new features and improvements so, everything "unrelated" with the release, is given the "integration_held" label. That means that will be ignored unless there is an unhold request process started on them.
Dev leaders promptly vote on held issues labelled with unhold_requested label and decide whether they can be allowed to break the freeze.
|3.||✓||Prepare a fresh installation of the QA site based on the beta code to make sure the site is not affected by incremental upgrade steps during the development cycle.||QA Master Site Maintainer|
|4.||✓||Begin reviewing new and changed English language strings ready for en_fix to be merged 2 weeks prior.||English fixes lang pack maintainer|
4 weeks prior
|1.||✓||Start reviewing fixed issues and add the release_notes label to all issues to be listed in the release notes.||Development Manager|
|2.||✓||Create issues for adding User tours for new features in the upcoming version. Review existing tours and remove very old ones||Development Manager|
|3.||✓|| Beta release (The Moodle release tool - mdlrelease - manages it): Release normal weeklies but with these changes in the master branch being considered all the time, until we are feature complete (can happen @ -4w - ideally - but also later):
|4.||✓||Clone MDL-68459 for the X.Y release, adding there all the debugging / PHP notices happening in the web server logs while running tests.||Integration Lead|
|5.||✓|| Release candidates release (The Moodle release tool - mdlrelease - manages it): At some points (between beta to final release) produce release candidates (Z = 1, 2, 3..), which are normal builds with the following changes:
|6.||✓|| Add a new version in the Plugins Directory with the version's name and the beta release version build number (https://moodle.org/plugins/admin/softwareversions.php).
|7.||✓||Push any plugins to the plugins database which were previously a part of core.||Plugins Liaison|
|8.||✓||Tidy up current latest en version of Moodle Docs prior to copying it to create new version wiki as described in New docs version process.||Community Manager|
|9.||✓||Create new QA test cycle and post in moodle.org front page news forum about QA testing.||Testing Maintainer|
|10.||✓||Invite community volunteers to start QA testing.||Community Manager|
|11.||✓||Modify Current QA cycle filter to link to the new QA test cycle||Testing Maintainer|
|12.||✓||Monitor QA fails. Check each fail is real and if so ensure an MDL issue has been created and correctly linked and labelled.||Testing Maintainer|
|13.||✓||Monitor MDL issues created for QA fails. Add them to the "Must fix for X.Y" list and get a developer to work on the issue immediately.||Development Manager|
|14.||✓||Review the list of features and improvements submitted before code freeze which are awaiting integration review and decide on related QA tests to be put on hold.||Development Manager|
3 weeks prior
|1.||✓||Ask developers to begin QA tests marked external_skipped.||Development Manager|
|2.||✓||Create new en and de Moodle Docs version wikis.||Moodle Docs Maintainer|
|3.||✓||Go through all points listed under 3 weeks prior in New docs version process.||Community Manager|
|4.||✓||Review issues with labelled with dev_docs_required, prompting assigned developers to create/update this documentation and remove this label when relevant docs are updated.||Development Manager|
|5.||✓||✓||Verify how everything is going and, before the end of the week, decide (dev leaders & integrators) if there are real reasons for delaying any release. Proceed to communicate any delay ASAP to both developers & partners. Don't forget to update calendars and tracker releases with the new dates.||Development Manager|
2 weeks prior
Check the number of open QA tests. Depending on the amount of work involved, encourage HQ developers to assist from the start of the week or later in the week in order to achieve the target of 100% pass rate by the end of the week.
Create MDLSITE issue to ensure all prototype.moodle.net sites which are no longer relevant (part of release) are removed from prototype.moodle.net
|3.||✓||✓||Merge fixes from en_fix pack and then integrate them.||AMOS Maintainer|
|4.||✓||Give Partners complete brandable Marketing pack for release including list and description of major new features.||Marketing Manager|
|5.||✓||Label (ui_change) in the Tracker all pending issues having noticeable UI modifications and track them in the "Late landing UI changes" sheet, in order to have all the associated docs, screenshots, videos under control. The Integration and Documentation people will try to keep the sheet updated until release.||Integration Lead|
1 week prior
|1.||✓||Clone as many filters as needed in the Tracker, modifying them to point to the new, upcoming, branch (keeping same perms, title...).||Integration Lead|
|2.||✓||✓||Create new minor version X.Y.Z+1 in the Tracker (MDL and CONTRIB). Archive any version > 6 months old, verifying that there aren't open issues using that version.||Integration Lead|
|3.||✓||Clone MDL-67199 and bump all versions, requires and dependencies along all plugins in codebase to planned release dates.||Integration Lead|
|4.||✓||✓||Post a "Heads-up" message on the General Developer forum.||Development Manager|
|5.||✓||✓||Post a "Heads-up" message on the Partners forum.||Security Officer|
|6.||✓||✓||Post a "Heads-up" message on Twitter and other outlets.||Marketing Officer|
|7.||✓||✓|| Identify security issues that need to be integrated using the security issues under integration filter (includes held, being integrated and ready issues).
|8.||✓||✓|| Collect security issues into a clone of the last advisory in the list to prepare for release of Security Advisories.
|9.||✓||✓|| Review and complete the release notes, making sure that issues have easily understandable summaries. Aim to have the release notes mostly complete on the Wednesday before release, and fully complete on the Friday, to give translators time to translate them.
|10.||✓||Check on the "Must fix for X.Y version". Filter out unrealistic issues.||Development Manager|
|11.||✓||Upgrade moodle.org to beta release|
|12.||✓||Upgrade moodle.org and all other Moodle community sites (next sites first, then production) - Friday before release if not before|
|13.||✓|| Prepare pull requests for CI Repositories:
|14.||✓||Go through all points listed under 1 week prior in New docs version process.||Community Manager|
This should happen immediately before the next integration cycle begins on Monday (i.e., some days after last weekly, 2 days prior to official release).
|1.||✓||✓|| Check list:
|2.||✓||✓||Verify all unit tests and integration tests have passed (check all current CI servers in use).||Integration Lead|
|3.||✓||Verify QA tests have passed.||Integration Lead|
|4.||✓||✓|| Timing prerequisites (don't continue with the process if not within the time window!):
(in practice, this means that the packaging of minor/major releases must be done, assuming that weeklies were rolled normally on Thursday, between Friday 09:00 and Monday 05:00 (Australia/Perth times). Breaking any of these rules will require manual operation in the server, forcing repackaging and releasing.
Follow the "mdlrelease" steps:
For major releases only:
Both with minors and majors, continue with the "mdlrelease" steps:
|5.||✓||✓||Post a "git repos updated & tagged" message on the Partner forum||Integration Lead|
|6.||✓||In the performance comparison repository, set the new release commit hash as $basecommit in the master branch and create a new MOODLE_XY_STABLE branch from it. (example).||Testing Maintainer|
|7.||✓||✓||Wait for the automated (every 2 hours) moodle-package to finish building for all versions. Verify the process has ended successfully (email).||Integration Lead|
|9.||✓||✓|| In the Tracker...
For major releases only:
|11.||✓||✓|| In the Tracker...
For major releases only:
For all releases:
|12.||✓||Clone MDL-67283 and MDL-67284, to be resolved ASAP.||Integration Lead|
|14.||✓||Protect the MOODLE_XY_STABLE branches in github interface to prevent non-FF accidents ( MDLSITE-4183 )||Integration Lead|
|15.||✓||The "integration_held" label will be removed only from bug issues awaiting integration; they correspond to last-week bugs that were held because of them being unrelated with the release. Now they can be processed, under on-sync rules. Standard message will be used to explain the un-hold.||Integration Lead|
Usually on Monday
|1.||✓||✓||The execution of "moodle-package-extract-and-postprocess X" script may be needed if the releases *are not going to be published on Monday* but another weekday (X is the weekday, (1-7) starting in Monday). By default it happens automatically around 09:00 AM Perth/Australia time.||Integration Lead|
|2.||✓||✓||Verify release status, download pages and windows packages.||Integration Lead|
|3.||✓||✓||If needed, modify the downloads cfg script (serverscripts) to decide about branches status, master visibility and windows availability. Push changes, will be autodeployed in a few minutes.||Integration Lead|
|4.||✓||✓||Add/update the release date, build number and link on the Releases page and date in new version pages.||Integration Lead|
|5.||✓||✓||Notify all registered sys admins, including security notes with CVE identifiers, using the mailing list server.||Security Officer|
|6.||✓||Post about the Major release on the moodle.org News forum||Lead Developer|
|7.||✓||Post about minor releases on the moodle.org News forum||Development Manager|
|8.||✓||New "X.Y+1dev" branch to be created in AMOS, to have master changes performed there. New install_XY_STABLE processing set up. These needs to be done anytime during the on-sync period.||AMOS Maintainer|
|9.||✓||✓||Verify, 24h after tagging, that https://moodle.org/dev/contributions.php has been updated with new versions. If not, file an urgent mdlsite issue, crons must be running!||Integration Lead|
|10.||✓||For en and de Moodle Docs, update default redirects and enable email notifications.||Moodle Docs Maintainer|
|11.||✓||Go through all points listed under Day of release in New docs version process.||Community Manager|
|13.||✓||Add calendar events in the moodle.org calendar for coming Major and Minor releases up to the next Major release.||Community Manager|
|14.||✓||✓||Update release schedule image on Releases page||Development Manager|
1 week after
|1.||✓||✓|| Publish the X.Y+ packages for download.moodle.org (should be automatic once weeklies are packaged).
For major releases, within the on-sync period, master packages will be generated using the prerelease.sh script (--type on-sync). That guarantees XY_STABLE and master versions are kept the same. The last week, when leaving the on-sync period, master package will be generated normally (implicit --type weekly), so versions will diverge and dev can continue separately from that point.
|2.||✓||✓||Update the version.php in git to be X.Y.Z+ during the next weekly integration process||Integration Lead|
|3.||✓||✓||Create a new release notes page for the next minor versions (using the release notes template.) Add note to top of relevant version page about security support.||Development Manager|
|4.||✓||✓||Add all security advisories to Security news and release notes with links to security advisories||Security Officer|
|5.||✓||✓||Notify about publications of CVE using form: https://cveform.mitre.org/||Security Officer|
|6.||✓||Upgrade moodle.org and all other Moodle community sites (next sites first, then production)|
2 weeks after
|1.||✓|| Once the "On Sync" period ends:
|2.||✓|| In https://github.com/moodlehq/moodle-behat-extension project:
|3.||✓||Ensure Language pack for master (X.Y+1) is available and merge the pull request MR4 for stop skipping the language upgrade in moodle-ci-runner repository.||Testing Maintainer|
|4.||✓||Hold a release retrospective with the entire team.||Development Manager|
1 month after
|1.||✓||Remove, in CI servers, all the jobs and views corresponding to branches which support has ended completely. (there is a maintenance job to perform the operation).||Integration Lead|
|2.||✓||Upgrade all the Moodle CI sites to recent major release by configuring the "Moodle CI Auto Upgrade" job in all them.||Integration Lead|
|3.||✓||Confirm that there isn't any remaining integration_held issue from latest release, proceeding to uhhold them immediately. Note that there may exist others "held" issues, unrelated with latest release. This process step does not affect them.||Integration Lead|