This project contains the scripts used to prepare Moodle code for Major, Minor and weekly releases.
- About this tool
- Installation
- Update
- Release types
- Advanced release types
- After the release
- Pre-release options
- Advanced pre-release options
This tool helps you prepare a Moodle release. It includes details on the required steps and scripts to facilliate those steps. In producing a release you simple need to work through the steps.
As this involves pushing Moodle source code to the public git repositories we maintain you must ensure you have had your ssh keys added to the following repositories by an administrator.
- git://git.moodle.org/moodle.git
- git@github.com:moodle/moodle.git
- git@bitbucket.org:moodle/moodle.git
Before running releases this project needs to be set up. This only need to be done the very first time you use this tool. Set up is dead easy thanks to a the installation script:
git clone https://github.com/moodlehq/mdlrelease.git
cd mdlrelease
./install.sh protocol username
protocol: defaults to 'ssh', also accepts 'https'. username: ignored for 'ssh' (keys are used there), optional for 'https' (git will ask for it if not set).
For more info about the protocols and their differences, read the "Setup required for new iTeam member" article.
The installation script prepares a Moodle repository and the necessary branches. As this involves a Moodle checkout the process can take some time to complete. Please be patient.
These scripts rely on some GNU utilities, and macOS ships the BSD flavor of command line tools instead of the GNU flavor. Homebrew can be use to install the GNU versions of tools, as well as MacPorts.
Homebrew
You can install most of the GNU flavored tools with:
brew install autoconf bash binutils coreutils diffutils ed findutils flex gawk \
gnu-indent gnu-sed gnu-tar gnu-which gpatch grep gzip less m4 make nano \
screen watch wdiff wget
To use the GNU versions by default, append to your /.profile ({/.zprofile if you use zsh):
# Get list of gnubin directories
export GNUBINS="$(find /usr/local/opt -type d -follow -name gnubin -print)";
if type brew &>/dev/null; then
HOMEBREW_PREFIX=$(brew --prefix)
for d in ${HOMEBREW_PREFIX}/opt/*/libexec/gnubin; do export PATH=$d:$PATH; done
fi
Source: https://gist.github.com/skyzyx/3438280b18e4f7c490db8a2a2ca0b9da
MacPorts
To install the port:
sudo port install gnutls
Add them to your PATH:
PATH="/opt/local/libexec/gnubin:$PATH"; export PATH
There are several different types of release that can be made by this tool. In fact it covers all of the planned release types.
Weekly release - the most common release, made every week and containing all the newly integrated work. Includes main and all stables branches.
Minor release - every two to three months we make a minor release, this release type handles everything involved with that. Only stable branches are worked on here.
Beta release - Around a month or two before a major release we create a beta release of the main branch. This usually occurs when QA testing starts and only affects the main branch.
RC release - The release candidate release. Usually one or more release candidate releases are prepared for the main branch in the final lead up to the major release. Again main only.
Major release - twice a year we make a major release (and usually a minor release of stables at the same time). This affects only the main branch although it leads to the next stable branch being produced.
There are also a handful of advanced release types. These release types are usually very specific and you may - or may not be required to run them. As such they are unplanned and usually only required if the situation demands it.
The following steps must be followed to perform a weekly release.
0. Ensure that we have achieved true roll-ability
Only possible when:
- CI status is green (only controlled problems allowed).
- There aren't remaining issues under testing. That may imply that you have to pass some issues under automated testing on CiBoT's behalf.
1. Run the pre-release script.
./prerelease.sh
By default this script prepares the branches and gives you the commands to push. It doesn't actually push up to the integration server. there is an optional argument -p which if specified pushes the updated branches to the integration repository. This option is useful if you trust the script more than you should ;)
2. Review the branches very carefully.
Note that weekly rolls involve, exclusively, supported branches. Always. No distinction the weeks before releases or the on-demand/on-sync explained below.
Only exception is when, after agreement, it has been decided to force something to land into unsupported branches and configuration is altered to cause that (for example environment.xml changes case).
3. Push changes to the integration repository
Use the command provided by prerelease.sh to push these last changes to the integration repository.
4. Confirm that all is green
http://integration.moodle.org jobs chain will start once last version bump is pushed to integration. Wait until all tests finish to confirm that all branches are ready to be public.
5. Push to public repository
Spread changes in integration to moodle.git and mirrors using ./release.sh (you may need, on releases, to comment some branches if not releasing all them together).
6. Complete remaining tasks
Follow the After the release steps where needed.
Note: Minor releases are usually produced on the weekend, a normal weekly is produced for stable supported branches on the weekdays before it.
The following steps must be followed to perform a minor release. The minor release affects all stable branches, main however is skipped as you cannot produce a minor release on an unreleased branch.
1. Run the pre-release script.
./prerelease.sh --type minor
By default this script prepares the branches and gives you the commands to push. It doesn't actually push up to the integration server. there is an optional argument -p which if specified pushes the updated branches to the integration repository.
2. Review the branches very carefully.
3. Push changes to the integration repository
Use the command provided by prerelease.sh to push these last changes to the integration repository. Don't push tags at this stage.
4. Confirm that all is green
http://integration.moodle.org jobs chain will start once last version bump is pushed to integration. Wait until all tests finish to confirm that all branches are ready to be public. Once verified and passing, push tags following the commands provided by prerelease.sh.
5. Push to public repository
Spread changes in integration to moodle.git and mirrors using ./release.sh (you may need, on releases, to comment some branches if not releasing all them together).
6. Complete remaining tasks
Follow the After the release steps where needed. Then, continue with the Moodle release process for major versions.
The following steps must be followed to perform a beta release. The beta release puts the main branch into a beta state. This usually happens in conjunction with QA testing, however it can happen any time we feel the main branch is maturing + stabilising in the lead up to a major release. Stable branches are skipped of course.
0. Ensure that we have achieved true roll-ability
Only possible when:
- CI status is green (only controlled problems allowed).
- There aren't remaining issues under testing. That may imply that you have to pass some issues under automated testing on CiBoT's behalf.
1. Run the pre-release script.
./prerelease.sh --type beta
Note that we might want to release stable branches together with a main beta, if that is the case we use the following command instead:
./prerelease.sh --type weekly
# Overwrite weekly main branch with a main beta.
./prerelease.sh --type beta
By default this script prepares the branches and gives you the commands to push. It doesn't actually push up to the integration server. there is an optional argument -p which if specified pushes the updated branches to the integration repository.
2. Review the branches very carefully.
3. Push changes to the integration repository
Use the command provided by prerelease.sh to push these last changes to the integration repository.
4. Confirm that all is green
http://integration.moodle.org jobs chain will start once last version bump is pushed to integration. Wait until all tests finish to confirm that all branches are ready to be public.
5. Push to public repository
Spread changes in integration to moodle.git and mirrors using ./release.sh (you may need, on releases, to comment some branches if not releasing all them together).
6. Complete remaining tasks
Follow the After the release steps where needed.
The following steps must be followed to perform a release candidate release. There can be one or more release candidate releases made as the final build up to a major release. They signify that we believe main branch is now stable and that we don't expect to find any more significant issues before release. We have usually addressed all QA related issues and release blocking issues. Again main only.
0. Ensure that we have achieved true roll-ability
Only possible when:
- CI status is green (only controlled problems allowed).
- There aren't remaining issues under testing. That may imply that you have to pass some issues under automated testing on CiBoT's behalf.
1. Run the pre-release script.
./prerelease.sh --type rc 2
Where 2 is the release candidate version.
Note that we might want to release stable branches together with a main RC, if that is the case we use the following command instead:
./prerelease.sh --type weekly
# Overwrite weekly main branch with a main RC.
./prerelease.sh --type rc 2
By default this script prepares the branches and gives you the commands to push. It doesn't actually push up to the integration server. there is an optional argument -p which if specified pushes the updated branches to the integration repository.
2. Review the branches very carefully.
3. Push changes to the integration repository
Use the command provided by prerelease.sh to push these last changes to the integration repository.
4. Confirm that all is green
http://integration.moodle.org jobs chain will start once last version bump is pushed to integration. Wait until all tests finish to confirm that all branches are ready to be public.
5. Push to public repository
Spread changes in integration to moodle.git and mirrors using ./release.sh (you may need, on releases, to comment some branches if not releasing all them together).
6. Complete remaining tasks
Follow the After the release steps where needed.
The following steps must be followed to perform a major release. Hurrah - we are ready for a major release, a twice yearly occasion normally. This will take the main branch only, produce a major release, and then create the next stable branch. After running this release type you will be required to edit these scripts and add a reference to the newly created stable release so that it is included in the processes here-after.
1. Run the pre-release script.
./prerelease.sh --type major
By default this script prepares the branches and gives you the commands to push. It doesn't actually push up to the integration server. there is an optional argument -p which if specified pushes the updated branches to the integration repository.
2. Review the branches very carefully.
3. Update the mdlrelease scripts
Add support for the new branch in install.sh, prerelease.sh, and release.sh
4. Prepare the current integration server
Create a new repo, view and jobs (cloning from main) in the Jenkins servers, so the new branch becomes tested by 1st time.
5. Double check everything.
Verify integration.git looks 100% perfect before continuing
6. Push to the public repositories
Spread changes in integration to moodle.git and mirrors using ./release.sh (you may need, on releases, to comment some branches if not releasing all them together).
6. Complete remaining tasks
Follow the After the release steps where needed. Then, continue with the Moodle release process for major versions.
The following advanced release types are also available.
Used to produce an on-demand release for the main branch. This type of release skips all stable branches. The following steps must be followed to perform an on-demand release.
0. Ensure that we have achieved true roll-ability
Only possible when:
- CI status is green (only controlled problems allowed).
- There aren't remaining issues under testing. That may imply that you have to pass some issues under automated testing on CiBoT's behalf.
1. Run the pre-release script.
./prerelease.sh --type on-demand
Note that most of the time we also release stable branches together with a main on-demand, if that is the case we use the following command instead:
./prerelease.sh --type weekly
# Overwrite weekly main branch with an on-demand main.
./prerelease.sh --type on-demand
Also, note that every on-demand release adds unconditionally the plus ('+') at the end of the version, so these changes are expected and normal: dev => dev+, beta => beta+, rc1 => rc1+ ...
By default this script prepares the branches and gives you the commands to push. It doesn't actually push up to the integration server. there is an optional argument -p which if specified pushes the updated branches to the integration repository.
2. Review the branches very carefully.
3. Push changes to the integration repository
Use the command provided by prerelease.sh to push these last changes to the integration repository.
4. Confirm that all is green
http://integration.moodle.org jobs chain will start once last version bump is pushed to integration. Wait until all tests finish to confirm that all branches are ready to be public.
5. Push to public repository
Spread changes in integration to moodle.git and mirrors using ./release.sh (you may need, on releases, to comment some branches if not releasing all them together).
6. Complete remaining tasks
Follow the After the release steps where needed.
Used to produce an on-sync release for the main branch. This type of release skips all stable branches. This release type should only be used when the Moodle's main branch must stay "in sync" with the latest stable branch after a major release. Note that the last week of the period, when on-sync ends, it's better to perform a normal main release (weekly) in order to guarantee that versions have diverged and avoid potential problems. The following steps must be followed to perform an on-sync release.
0. Ensure that we have achieved true roll-ability
Only possible when:
- CI status is green (only controlled problems allowed).
- There aren't remaining issues under testing. That may imply that you have to pass some issues under automated testing on CiBoT's behalf.
1. Run the pre-release script.
./prerelease.sh --type on-sync
Note that most of the time we also release stable branches together with a main on-sync, if that is the case we use the following command instead:
./prerelease.sh --type weekly
# Overwrite weekly main branch with an on-sync main.
./prerelease.sh --type on-sync
By default this script prepares the branches and gives you the commands to push. It doesn't actually push up to the integration server. there is an optional argument -p which if specified pushes the updated branches to the integration repository.
Note that the last week of on-sync, it's better to perform a normal main release (weekly) in order to guarantee that versions have diverged. If this is such a week, please proceed accordingly.
IMPORTANT: If this is the last week of on-sync, don't forget to disable the Continuous queues manager job right now to prevent it to continue processing issues once on-sync is finished.
2. Review the branches very carefully.
3. Push changes to the integration repository
Use the command provided by prerelease.sh to push these last changes to the integration repository.
4. Confirm that all is green
http://integration.moodle.org jobs chain will start once last version bump is pushed to integration. Wait until all tests finish to confirm that all branches are ready to be public.
5. Push to public repository
Spread changes in integration to moodle.git and mirrors using ./release.sh (you may need, on releases, to comment some branches if not releasing all them together).
6. Complete remaining tasks
Follow the After the release steps where needed.
Used to produce a back-to-dev release after a major release. This release type should only be used immediately after a major release has been successfully completed in order to set the main branch back to a development state. The following steps must be followed to perform an on-sync release.
1. Run the pre-release script.
./prerelease.sh --type back-to-dev
By default this script prepares the branches and gives you the commands to push. It doesn't actually push up to the integration server. there is an optional argument -p which if specified pushes the updated branches to the integration repository.
2. Review the branches very carefully.
3. Push to the public repositories
Spread changes in integration to moodle.git and mirrors using ./release.sh (you may need, on releases, to comment some branches if not releasing all them together).
You're not quite done yet, you must do the following after all releases.
0. Verify external testing counters
Note: this only applies to releases where real issues have been fixed, usually weeklies, on-demand, on-sync, not for "tagging" releases like betas, rcs, minors, majors (unless they come with any new issue fixed).
Before closing the issues in the Tracker and everything else, please visit the Sorted testing assignments sheet and verify that the numbers in the "External" tab have been updated for the current cycle.
If not, and the cycle does not exist yet, fill both the Week and Date (start of Week) in a new row (bottom-up) and press the "Update empty week row" button. That will fetch all the information and fill the row properly.
If they were just outdated but the row already existed, you can simply clean the pink cells contents and press the button in order to get the information fetched.
1. Update our records.
Note: for all release types, no matter the counters are zero.
Annotate the number of closed, reopened and queued issues in the sheet. Note: The queues total is the sum of all the issues in both (candidates and current) queues. Usually, as a cell comment in the sheet, we add the details about: Awaiting candidates / awaiting current / integration in progress / waiting for feedback.
2. Tidy up tracker.
Note: for all release types, as far as there are issues to close.
Run the Close tested issues job in the CI server. It will close all the tested issues under current integration (you can provide an alternative date or comment there). It will perform these changes:
- Status: Closed
- Change resolution: Fixed
- Change integration date: Today's date
- Change currently in integration -> None
- Default thanks message.
Note: If there is any problem with the job, still it's possible to proceed using Tracker's bulk actions (transitioning to closed). If using this, you can temporarily disable the autowatch user preference to avoid autowatching all those issues.
Don't forget to review any closed issue having "mdlqa" or "mdlqa_conversion" label. They may need resetting or moving associated MDLQA issues. There is a widget about this in the integration Dashboard too.
3. Spam the tracker.
Note: Only under normal integration periods. Aka, not under continuous integration.
Run the Send rebase message job in the CI server. It will send the standard rebase messages to all issues awaiting for integration for the next week (you can provide an alternative comment using altcommentcandidate). It also sends the message to issues in current queue waiting for integration or in progress (in that case, you can use the altcommentcurrent field to provide an alternative comment).
Note: If there is any problem with the job, still it's possible to proceed using Tracker's bulk actions (sending a comment). If using this, you can temporarily disable the autowatch user preference to avoid autowatching all those issues.
4. Let the world know.
Note: Only under normal integration periods. Aka, not under continuous integration. Unless there is something relevant enough and agreed to be shared.
Add one entry to the "Integration, exposed" forum, commenting about numbers, major ones, special thanks... always checking the "Integration exposed ideas" document for both general and policy issues to be shared or proposed, together with other interesting topics worth commenting and thanks suggestions.
The following options can be used to customise how the selected release type is prepared.
-
-b | --branch
Limits the operation to just the branch that has been given. By default the appropriate branches for the release type will all be operated on.
-
-d | --date
Enforces a build date for all the branches being processed. The use of this option overrides the default behavior, that is the following:
- "next monday" is used for major and minor releases.
- "today" is used for any other release type.
-
-n | --not-forced
By default the version file on all branches will be bumped. If this option has been specified then the version file will only be bumped if there are new commits on the branch
-
-p | --pushup
By default this script prepares everything to be pushed by does not push. If this option is specified the staged commits and any tags will be pushed up to the integration server.
-
-q | --quiet
If set this script produces no progress output. It'll let you know when its finished however.
-
-t | --type
The type of release to prepare.
-
--no-create
If this tool finds that one of the expected branches does not exist then by default it creates it. If this option is specified the tool will not create the branch but will exit with an error.
The pre-release script has several arguments that can set to alter how the release is prepared, what steps are taken, and other functions to aid the integrator in the release process.
For detailed information on these options you can already run the following from your console:
./prerelease.sh --help
This resets the expected branches in the local repository.
./prerelease.sh --reset
Please be aware that it doesn't actually fetch upstream changes, it simply resets the expected branches to the currently tracked state. This is a particularly useful option if you've naughtily being hacking on the mdlrelease repository.
Note: this is an exclusive function, the prerelease script exits immediately after the reset operations have been completed.
This displays the state of the local repository, making it clear if there are any changes that would get in the way of preparing a release.
./prerelease.sh --show
It also gives you the option of viewing the changes that have been made on a branch that is not in-sync with its tracked upstream branch.
Note: this is an exclusive function, the prerelease script exits immediately after the show operations have been completed.