petervanderdoes/gitflow-avh

[Question] git flow possible incorrect versioning?

CrispyDrone opened this issue · 5 comments

CrispyDrone commented

In GitFlow, are you allowed to merge features into develop once a release branch has been created?

As far as I can tell this seems to result in incorrect versioning of certain commits, namely:

  • All the commits that are added to develop before the release branch gets merged back with its tag will be reported as "belonging" to the previous version according to git describe. git describe will find the newly created tag for any commits that are created after merging the release branch with develop. But any commits before it got merged back and after the forking from the release branch will show the previous tag even though they're not part of this release, but the next one.

  • These same sets of commits will have an incorrect version number in any files such as AssemblyInfo, package.json, or any other files that contain a version number. This is because git flow only mentions bumping a version number inside the release branch. When merging back the release branch, I'm assuming that it will take over the version number of the release branch.

    So all commits before the release branch was started will have the previous version number. All commits after the release branch will have the next version number. But all the commits in between, will also have the previous version number.

Is this actually an issue? If not, please explain why. If yes, how do you deal with it?

At work, we've been experiencing a real issue related to this.

For the sake of the example, imagine 2 repositories, both using git flow.
Repository A consumes an artifact published from repository B (nuget package).
Repository B is currently at version 1.0.0 in master, and version 1.1.0-alpha+3 in develop.

Now the following sequence of actions occurs:

  1. Repository B creates a release branch from 1.1.0-alpha+3. Meanwhile development continues, and two additional features are merged with develop. Repository B now has version 1.1.0-beta+0 in the release branch, while develop is already at 1.1.0-alpha+5.
  2. Repository A takes a dependency on 1.1.0-alpha+5 version of repository B that was created after the release branch in repository A was started.
  3. Repository B's release branch is finished, and a stable package version 1.1.0 is created.
  4. Repository A's dependency on 1.1.0-alpha+5 is upgraded to the stable equivalent 1.1.0.
  5. BOOM: 1.1.0 does not contain the 2 additional features 1.1.0-alpha+4 and 1.1.0-alpha+5.

I guess there are multiple ways around this problem:

  1. Only consume stable packages. However, in our current environment this is not realistic as we have major release cycles.
  2. Work with alpha/beta versioning for both the file versions and git tags. By doing this, any new features merged into develop after the release branch has been started will be working towards the "next next" version. In the example that would result in 1.2.0-alpha+1 and 1.2.0-alpha+2 instead of 1.1.0-alpha+4 and 1.1.0-alpha+5.
talios commented
CrispyDrone commented

Thank you for your reply @talios. Sorry for my late response.

Could you explain a bit more what you mean with:

However, when you create your release branch, you’d update the version to
1.1.0-beta-1-SNAPSHOT, which during the release process would update that
version further to 1.1.0-beta-1 - and tag the repository with that
specific commit, deploy that to nuget etc.

Is 1.1.0-beta-1-SNAPSHOT reflected in the version number that's typically
stored in a file like package.json or AssemblyInfo.cs? Next, the CI system
simply uses this version and optionally attaches some build metadata to it?

So when I'm about to finish my release branch, is it tagged with 1.1.0-beta-1
or 1.1.0? If the former, how does master now get the 1.1.0 tag? Is this
possible with the gitflow extensions of this repository?

…then finally update the version to be 1.1.1-alpha-1-SNAPSHOT which gets
merged back into develop (by way of master) for any subsequent development
builds.

This update to 1.1.1-alpha-1-SNAPSHOT is done in the release branch? You
merge back master with develop? In the original git flow
model it is
described that you merge release back into develop, not master into develop?
Also, I presume you meant 1.2.0-alpha-1-SNAPSHOT instead of
1.1.1-alpha-1-SNAPSHOT?

Now to further clarify my situation. We don't explicitly version anything in
our local repositories. git describe and the files that contain version
numbers are examples that I used to highlight the potential issue that I think
exist in how git flow operates or how certain tools make assumptions about
versioning.

My current understanding of git describe is that you should be able to rely
on it to give you the previous version of your repository from the perspective
of a random commit. But, I just realized that there's actually no reason for
that commit to then automatically have to be included in the "next release".
This depends entirely on the branching model. For example in Github Flow, it
could be that a feature branch doesn't get merged for a very long time, and
until you would do a rebase it would naturally report a much older version
number, and at any point in time when you check this, there's no guarantee it's
going to be included in the "next release".

However with git flow there is no such concept. Any new development on the
develop branch must get included in the next release. So if my previous
assumption is correct, then in theory git checkout develop && git describe --abbrev HEAD should always give you the version number that belongs to the
previous release, and any commits that report this version number must
belong to the next release.

To come back to the tooling, the setup we currently have is entirely automated
through the git flow extension and a CI task known as
GitVersion. Our "release manager" is actually
responsible for everything except for creating features and hotfixes (don't ask
me why, we are trying to improve the situation desperately).

So us developers, we don't care about versions numbers at all in our local
repos. We simply create feature branches and merge them with develop.

GitVersion automatically attaches a version number to every build our CI server
does. The information after + is actually build metadata which for develop
only includes the build number, it's not the number of commits past a git tag
as reported by git describe, apologies if I confused you. If someone
builds their feature branch, it will also include the feature name.

We do not translate every build of repository B into a release of its
nuget package. This could be done, but we don't have this automatic trigger
configured.

Instead developers release a version when they need it. So for example, a
developer is working on a new "feature" that requires modifying repository A
and B at the same time. They now need to consume their new development in
repository A so they simply publish a nuget package from whatever artifact has
their development—in this case that was 1.1.0-alpha+5. This developer will
merge their work with the develop branch in repository A. Some time later when
1.1.0 is published for repository B, there is the danger that anybody decides
to upgrade the non-stable package of repository B to its stable equivalent. But
this will blow up due to the lack of the 2 required features (introduced in
1.1.0-alpha+4 and 1.1.0-alpha+5).

For me personally, the thing that makes the most sense is that as soon as you
create your release branch, any contributions on develop already have to be for
the "next next" release. In the original git flow article that doesn't seem to
be described, and the git flow extensions here don't seem to promote that
either.

What I really want is 2 things:

  • every version number (local, or not) should refer to a unique, indentifyable
    binary and the source code that produced it
  • Updating from prerelease to release versions should be a completely safe
    operation

It would be of real help if we there was a guide somewhere that achieves this
and describes it it in terms of:

  • version files (package.json, AssemblyInfo.cs,...): when to update them
    and how
  • version tags: when to add git tags, with what versions (does this even make
    sense if you are using a monorepository)
  • build numbers on CI servers
  • package versions (nuget, maven,...)
ChrisJStone commented

For me personally, the thing that makes the most sense is that as soon as you
create your release branch, any contributions on develop already have to be for
the "next next" release. In the original git flow article that doesn't seem to
be described, and the git flow extensions here don't seem to promote that
either.

The nature of how branches work in git this is all ready the case. As long as you do not merge any changes from the release branch into develop prior to release you could consider develop to be in a code freeze state in regards to the status of release.

From A successful Git branching model

"Release branches support preparation of a new production release. They allow for last-minute dotting of i’s and crossing t’s. Furthermore, they allow for minor bug fixes and preparing meta-data for a release (version number, build dates, etc.). By doing all of this work on a release branch, the develop branch is cleared to receive features for the next big release."

In regards to

every version number (local, or not) should refer to a unique, indentifyable binary and the source code that produced it

For your release branches this is all ready taken care of for your source code. As part of git flow release finish the master branch is tagged with your release tag.
Once again from A successful Git branching model

"When the state of the release branch is ready to become a real release, some actions need to be carried out. First, the release branch is merged into master (since every commit on master is a new release by definition, remember). Next, that commit on master must be tagged for easy future reference to this historical version. "

version files (package.json, AssemblyInfo.cs,...): when to update them and how

You update them as often as you find necessary. You could use a text editor such as vim or write a script file to parse an environment variable.

version tags: when to add git tags, with what versions

Same as above. Create tags as often as you find necessary. Once again A successful Git branching model does a fantastic job of explaining some of the different branches used and the actual git commands needed to create branches and tags.

build numbers on CI servers

That's usually covered in the docs for your CI server.

In the end it is up to you and your development team to design the workflow that you are going to use. IE when to create branches, when to tag, when to merge how to merge etc...

CrispyDrone commented

Thanks for your response @ChrisJStone

I think that first quote is taken a bit out of context, or maybe I wasn't clear
enough in my wording. New developments on the develop branch after a release
branch has been created are indeed for the "next next" release. That is great.
The problem for me is that nor the git flow extensions of this repository, or
the original document are clear for me in regards to how versioning is
supposed to work in this scenario.

As mentioned in the section "creating a release branch" of the original
article:

After creating a new branch and switching to it, we bump the version number.
Here, bump-version.sh is a fictional shell script that changes some files in
the working copy to reflect the new version. (This can of course be a manual
change—the point being that some files change.) Then, the bumped version
number is committed.

This means the version number in develop remains unaffected. What happens to it
is not described.

As soon as you merge the release branch back into develop, these files that
were updated by the bump-version.sh will get updated. From the perspective of
the develop branch, the version in these files is completely unreliable now
because any new developments that occurred before the release branch was merged
back will still have the previous version even though they didn't get released
with other commits that shared that same version. This is part of the problem
I've been describing (the same problem exist for "git describe" in my opinion).

So based on this, and from the original description of git flow, I presume that
the intention is that develop doesn't have any "explicit versioning", which is
supported by:

It is exactly at the start of a release branch that the upcoming release gets
assigned a version number—not any earlier. Up until that moment, the develop
branch reflected changes for the “next release”, but it is unclear whether
that “next release” will eventually become 0.3 or 1.0, until the release
branch is started. That decision is made on the start of the release branch
and is carried out by the project’s rules on version number bumping.

However, if develop doesn't have a well-defined version or a version at all,
that makes it very impractical in my current situation as we have large release
cycles and often need to be able to take dependencies on software packages that
are not release candidates or stable (refer to my previous posts for an
example). That of course means that any package produced from develop would
need a well-defined version.

As for the possible solutions:

  • Only consume stable packages -> not an option for us in our current
    environment, perhaps in the future

  • Tag the commit that is the branching point of the release branch -> this is
    what we're currently looking into.

  • Don't assume you can safely upgrade a prerelease package to its stable
    version -> this goes against one of my principles, and would result in
    confusion about package upgrades at the consuming side.

The reason for opening this issue of course was to be able to understand how
other people are dealing with this issue, and if it's not an issue in their
workflows, how is that so?

Also, maybe I'm wrong in this sentiment, but I feel that a branching model and
versioning should be pretty much plug-and-play. So I don't really agree stating
that these are completely up to me/my team. I would say that the existence of
these git-flow extensions, and tools such as GitVersion are a
testament to that belief.

So since git-flow can automatically create tags based on the release branch
name when finishing a release, maybe it should do something similar when
starting a release? Concretely, I was looking more into GitVersion
and they actually seem to recommend something of the nature of tagging the
release branch with an "alpha" tag (however, I'm not sure if they actually mean
immediately tagging it as soon as you create your release branch. I hope
so, and will try to get some clarifications from them):

Note that at least one version tag is required on the branch. The recommended
initial tag is {releaseVersion}.0-alpha1. So for a branch named release-1.2
the recommended tag would be 1.2.0-alpha1

ChrisJStone commented

Your welcome @CrispyDrone, I may have.

However, if develop doesn't have a well-defined version or a version at all,
that makes it very impractical in my current situation as we have large release
cycles and often need to be able to take dependencies on software packages that
are not release candidates or stable

If you have a well defined versioning policy in place that is not a problem. I'm going to assume that you have a understanding of semantic versioning. With X.Y.Z being Major.Minor.Patch now you have also have a number of addition qualifiers such as ALPHA, BETA, RC, DEV, SNAPSHOT etc..... along with a build number, timestamp or what ever additional metadata your team decides is necessary to identify a unique version.

In my case I prefer to gpg-sign various commits to my project repository. So I typically do the following when I setup a new git repository.

I will perform a git init followed by git commit --allow-empty --gpg-sign. This way I can create the master branch and I know every commit made to master is signed by me.
After this is when I run git flow init. I have not bothered to do so but you can modify the git-flow scripts so that it will perform a --gpg-sign.

Once all of my initial setup is completed I will create an initial release. I usually will use 0.0.1 for this release. It really has no functionality but it's a starting point for development.

At this point I would have 2 commits on my master branch. With the second commit having the TAG of 0.0.1 and X number of commits on my DEV branch.

Now at the start of my changelog I would have a version of #### 0.0.1-DEV1 <--- I use this to track my versions
As commits are made to my dev branch I will increment my version with the number of commits made since the last merge from master. When I make a release from dev I will create a SNAPSHOT-# tag for example 0.0.1-SNAPSHOT-1

So I can have a repository version of 0.0.1-DEV4 with a release version of SNAPSHOT-1 and later on 0.0.1-DEV20 and SNAPSHOT-2

When I create a release such 0.1.0 I will update the version in dev to 0.1.0-DEV1 <-- since this is the start of a new development cycle

and the version in my release branch to 0.1.0-RC1 with RC being incremented as final bug fixes etc are made. Once a final release is made I commit the release branch to master and develop along with creating tag 0.1.0

My hotfix branches would be done similarly only I would start with 0.1.1 as my initial hotfix branch.

This particular fork of git-flow supports a number of hooks that could help you with what you need to accomplish. However you need to write the actual scripts. Now there are a couple of repos that @petervanderdoes has here that provide some example scripts. One of which he used while he was writing this fork of git-flow so it is setup for this repo.

I would say that the existence of these git-flow extensions, and tools such as GitVersion are a testament to that belief.

The whole point of git-flow and GitVersion is to help you the developer be more productive in your job. Instead of having to remember and type out multiple git commands to create and finish a release for your project all you have to type is git flow release start and the name of your branch and git flow release finish to finish the branch. It is still up to you to decide what to name that branch and the tag that is created.

I feel that a branching model and versioning should be pretty much plug-and-play.

If that were the case then why are there so many different VCS packages or CI servers. I've thought the same thing. In the end sadly it falls on us developers. With the number of different CI servers, VCS packages even development methodologies it's really not possible. The best thing that we can do is have a though understanding of how the software works and either find an existing tool that does what were looking for, find something close and write glue code to make it work within our development life cycle or write the tool ourselves.

There are software engineering standards that you can follow. However even in the standards it clearly stats that it is up to the end user of those standards to customize them to fit the individual needs.

Do you or the company you work for have a document that clearly states how your development and release life-cycles will work? What tools will you use, how will the release be versioned?

With out knowing exactly what the problem is we don't know what tool to use.