Metadata for 0.2.13 release has an issue.

Question

Metadata for 0.2.13 release has an issue.

MrJoy opened this issue 4 years ago · 10 comments

The changelog doesn't say anything about 0.2.13 being restricted to older versions of simplecov, but according to Rubygems the dependency went from >= 0 for codecov 0.2.12 to '~> 0.18.0` for 0.2.13. As such, those of us on the latest simplecov can't update.

Is this an oversight, or is there a compatibility issue that's been discovered with newer simplecov?

Answer 1 · 2020-12-16T11:30:13.000Z

Same here, trying to use both latest
simplecov (0.20.0)
and
codecov (0.2.13)
and seems this is not possible

Answer 2 · 2020-12-16T12:11:33.000Z

Hello there, restriction was made by me.

The changelog doesn't say anything about 0.2.13 being restricted to older versions of simplecov

Sorry, but there are no contributing guides with requirement "add changelog entries in PRs", so I just made changes and left changelog to maintainers. I personally update changelog only before release, also because some changes can be reverted before release.

but according to Rubygems the dependency went from >= 0 for codecov 0.2.12 to '~> 0.18.0` for 0.2.13.

Yep, >= 0 is very dangerous.

Is this an oversight, or is there a compatibility issue that's been discovered with newer simplecov?

At least there was not 0.20 version at PR creation date. Why there is 0.18 and 0.19 — I don't know, I think I saw an issue about incompatibility, but I can't find it right now.

And now description and and resolution: here is #115. And this is how we should track new versions of dependencies (including simplecov), see CI results, and then update (allow new ones). I suggest you to:

rollback codecov with simplecov >= 0, but it's dangerous;
or use an older version of simplecov (0.18) with current codecov and wait for a new release with #115 merged. Then update them all.

Answer 3 · 2020-12-16T13:51:08.000Z

Yep, >= 0 is very dangerous.

I think better solution is
s.add_dependency 'simplecov', '~> 0'

This allow to install any 0.x.x version of simplecov and if simplecov using Semantic Versioning this will not result a breaking changes

But with your soltuion

  s.add_dependency 'simplecov', '>= 0.18', '< 0.21'

You will need to release new version of codecov gem EACH time simplecov release a new version

You said several time that open requirements are dangerous, but I'm not pretty sure why it is. It's not like this tool has some real critical impact on application, can contain some security issues or something like this. It's just a toole to test code coverage, even if some upgrade will broke test coverage any user can revert upgade it back without breaking it's main app

Answer 4 · 2020-12-16T14:52:42.000Z

This allow to install any 0.x.x version of simplecov and if simplecov using Semantic Versioning this will not result a breaking changes

If you'll look at SemVer closer, you'll see that minor versions in zero major version backward incompatible: https://semver.org/#spec-item-4

Major version zero (0.y.z) is for initial development. Anything MAY change at any time. The public API SHOULD NOT be considered stable.

Zero major version means "unstable", "alpha version". And this is it.

You will need to release new version of codecov gem EACH time simplecov release a new version

Yes, I know, and it's OK. We don't see the future, we can't know if simplecov 0.21 or 0.51 will change their interface, on which Codecov relies, completely or especially partially. We should check every minor update in 0-major version like every major versions > 1. At least, we trust patch versions. After simplecov 1.0 release we should trust minor versions and change lock to ~> 1.0. But it's still under alpha-development, and we have to deal with it.

You said several time that open requirements are dangerous, but I'm not pretty sure why it is.

I don't know about this case, codecov and simplecov, but I suppose that you can't use the latest version of codecov with the first version of simplecov. Even if you can — it doesn't guarantee that you will. Imagine some web-application, for example, with dependency rails >= 0 (1, 2, 3, 4, 5, 6, 10, 90). This is wrong.

It's not like this tool has some real critical impact on application, can contain some security issues or something like this. It's just a toole to test code coverage, even if some upgrade will broke test coverage any user can revert upgade it back without breaking it's main app

I understand that this is not runtime dependency and not for production environment, but:

It can be unfriendly for newcomers to countless number of projects, using codecov, when they try to contribute and facing with errors, fails, etc. I don't think all of them want to discover unrelated problem.
Phantom CI fails, when you've done nothing, but they're failing, are annoying. Also some projects have branch protections, restrictions, that PRs before merge must pass all CI tasks. And you can't merge unrelated to these fails PRs until you'll fix it in upstream, like "yeah, the latest codecov is OK, but let's lock this new simplecov to the < 0.22 now, because 0.21 breaks our CI". Again, it can happen with any new release of any runtime codecov dependency in countless number of third-party projects, using codecov. And they'll have to do these locks manually, separately from each other. Let's keep them safely and do this lock in one place, when we check that everything works fine, and let's guarantee safe work (development, CI) for these projects which rely on us. I hope you understand. Yes, additional manual work. But in one place and with warranties for others.

Answer 5 · 2020-12-16T15:02:07.000Z

Zero major version means "unstable", "alpha version". And this is it.

A lot of ruby libraries has zero version as major version

After simplecov 1.0 release we should trust minor versions and change lock to ~> 1.0.

Is this going to happen at all?
https://rubygems.org/gems/simplecov/versions
Simplecov in development for 10 years and currently is 0.20.0

Answer 6 · 2020-12-16T15:15:10.000Z

Zero major version means "unstable", "alpha version". And this is it.

A lot of ruby libraries has zero version as major version

It's sad and it means that we can't rely on their minor versions. RuboCop as example.

Is this going to happen at all?
https://rubygems.org/gems/simplecov/versions
Simplecov in development for 10 years and currently is 0.20.0

I don't know, I suggest you to forward this question to simplecov maintainers. At least, I saw 1.0 releases recently from RuboCop and Faraday. And Crystal language 1.0 comes. So, it happens, for some projects at the first days, for other — after years or even never.

Still, if simplecov maintainers use 0 major version and not > 0, they mean "don't rely on our minor versions until we release 1.0", or they're wrong. It's still a question for them, not me.

Answer 7 · 2020-12-17T20:29:19.000Z

Sorry, but there are no contributing guides with requirement "add changelog entries in PRs", so I just made changes and left changelog to maintainers. I personally update changelog only before release, also because some changes can be reverted before release.

Gotcha. I only mentioned it because I wasn't sure if this was an intentional choice, or something that crept in by accident.

Yep, >= 0 is very dangerous.

Fair enough.

At least there was not 0.20 version at PR creation date. Why there is 0.18 and 0.19 — I don't know, I think I saw an issue about incompatibility, but I can't find it right now.

Seems possible the SimpleCov folks aren't following semver.

And now description and and resolution: here is #115. And this is how we should track new versions of dependencies (including simplecov), see CI results, and then update (allow new ones). I suggest you to:

Much appreciated!

rollback codecov with simplecov >= 0, but it's dangerous;

or use an older version of simplecov (0.18) with current codecov and wait for a new release with #115 merged. Then update them all.

For the time being, I've rolled back CodeCov because it was in an apparently-working state. I look forward to the next CodeCov release!

Answer 8 · 2020-12-17T23:28:15.000Z

For the time being, I've rolled back CodeCov because it was in an apparently-working state.

Please, let me know: it the latest release of CodeCov is broken? I think it just uses not the latest version of simplecov, but still works with 0.18 (in our case). What do you mean by "it was in an apparently-working state", if it still in this state in my understanding?

Answer 9 · 2020-12-18T00:43:20.000Z

SimpleCov 0.20.0 + CodeCov 0.2.12 was apparently working fine, so I rolled back to that instead of taking SimpleCov 0.18.x + CodeCov 0.2.13.

The current version of CodeCov seemed to work fine with the older SimpleCov, but I'm not interested in rolling back SimpleCov which is why I went that route.

Answer 10 · 2020-12-21T16:30:26.000Z

HI all, thanks for the incredible write-up and discussion. For now, we will be switching to

>= 0.15, < 0.21

The reason being as stated above, we cannot yet rely on non-breaking changes in future Simplecov versions. Case in point, the format changed for 0.18 and broke our ability to parse reports. We will continue to use this style of dependency until the first major version from Simplecov.

I'll be making the change and rolling out a new version of the Codecov gem this week.