renovatebot/renovate

gradle/libs.versions.toml replaces comment, not the version value

Closed this issue · 18 comments

TWiStErRob commented

How are you running Renovate?

WhiteSource Renovate hosted app on github.com

If you're self-hosting Renovate, tell us what version of Renovate you run.

No response

Please select which platform you are using if self-hosting.

No response

If you're self-hosting Renovate, tell us what version of the platform you run.

No response

Was this something which used to work for you, and then stopped?

I never saw this working

Describe the bug

gradle/libs.versions.toml has contents like:

[versions]
# GitHub releases: https://github.com/mockk/mockk/releases
# TODO can't go farther than 1.12.3 yet https://github.com/mockk/mockk/issues/794
mockk = "1.12.2"

[libraries]
mockk = { module = "io.mockk:mockk", version.ref = "mockk" }

renovate updates the comment, not the version number:

https://github.com/TWiStErRob/net.twisterrob.gradle.renovate.test/pull/4/files#diff-697f70cdd88ba88fe77eebda60c7e143f6ad1286bca75017421e93ad84fb87dfR55-R56

Relevant debug logs

Logs

https://app.renovatebot.com/dashboard#github/TWiStErRob/net.twisterrob.gradle.renovate.test/628749206

 {
            "depName": "io.mockk:mockk",
            "groupName": "io.mockk",
            "currentValue": "1.12.2",
            "managerData": {
              "fileReplacePosition": 2420,
              "packageFile": "gradle/libs.versions.toml"
            },
            "fileReplacePosition": 2420,
            "registryUrls": [
              "https://repo.maven.apache.org/maven2",
              "https://dl.google.com/android/maven2/",
              "https://plugins.gradle.org/m2/",
              "https://jcenter.bintray.com/"
            ],
            "depIndex": 44,
            "updates": [
              {
                "bucket": "patch",
                "newVersion": "1.12.3",
                "newValue": "1.12.3",
                "releaseTimestamp": "2022-02-28T13:53:38.000Z",
                "newMajor": 1,
                "newMinor": 12,
                "updateType": "patch",
                "branchName": "renovate/patch-io.mockk"
              }
            ],
            "warnings": [],
            "versioning": "gradle",
            "sourceUrl": "https://github.com/mockk/mockk",
            "homepage": "http://mockk.io",
            "currentVersion": "1.12.2",
            "isSingleVersion": true,
            "fixedVersion": "1.12.2"
          },

Have you created a minimal reproduction repository?

No reproduction, but I have linked to a public repo where it occurs

rarkins commented

Only happens if the comment also includes the same version?

TWiStErRob commented

Not sure, haven't tried, this is the first time I've seen this and haven't used renovate long enough.

rarkins commented

If this only fails if the comment includes the version then it's a low priority. If it fails for any comment at all then it's normal or high priority.

  • 👍1
PhilipAbed commented

I have tried to create an extraction test for the 2 cases, with the same version/not same version like below:

[versions]
# JUnit 4.13.1 is awesome!
junit4 = "4.13.1"

[libraries]
junit-legacy = { module = "junit:junit", version.ref = "junit4" }

And then for :

[versions]
# JUnit 4.13.2 is awesome!
junit4 = "4.13.1"

[libraries]
junit-legacy = { module = "junit:junit", version.ref = "junit4" }

they give the same replacement location, there's no difference at all,
After debugging, I can see the code is looking for junit4, then looking for 4.13.1 right after it.

I can conclude there's no problem in the extraction in the @TWiStErRob 's case:
https://github.com/TWiStErRob/net.twisterrob.gradle.renovate.test/pull/19/files#diff-697f70cdd88ba88fe77eebda60c7e143f6ad1286bca75017421e93ad84fb87dfR42-R43

but i still find the same issue could happen if i create my own case:

[versions]
# junit4 4.13.1 is awesome!
junit4 = "4.13.1"

[libraries]
junit-legacy = { module = "junit:junit", version.ref = "junit4" }

since the extraction is trying to find "junit4" and its in the comment.

The Update isn't the problem in this case either, since it updates depending on that replacement position offset
if the replacement position value is OK for @TWiStErRob then what could be the problem?

PhilipAbed commented

ok found the issue:
image

viceice commented

it seems the fileReplacePosition is wrong when searching the version after extracting from toml file. We should make sure we only search for =\s+"<version>" when searching the file position.

renovate/lib/modules/manager/gradle/extract/catalog.ts

Line 79 in f5f06c7

function extractLiteralVersion({

i don't like the file position replacement at all, as it currently causes bugs that following updates are replacing at wrong position.

PhilipAbed commented

yeah i agree,
ill add a unit test with a file like this:

[versions]
# Releases: http://someWebsite.com/junit/1.4.9
mocha-junit-reporter = "2.0.2"
# JUnit 1.4.9 is awesome!
junit = "1.4.9"


[libraries]
junit-legacy = { module = "junit:junit", version.ref = "junit" }
mocha-junit = { module = "mocha-junit:mocha-junit", version.ref = "mocha-junit-reporter" }

and try to refactor it to make it work as expected.

PhilipAbed commented

@viceice =\s+"" doesnt work for plugins :D other tests failed:

[plugins]
kotlinJvm = { id = "org.jetbrains.kotlin.jvm", version = "1.5.21" }
viceice commented

@viceice =\s+"" doesnt work for plugins :D other tests failed:

[plugins]
kotlinJvm = { id = "org.jetbrains.kotlin.jvm", version = "1.5.21" }

🤔 the version is quoted and there's an equal sign before. why it doesn't match?

PhilipAbed commented

im creating a more complex regex for all cases i guess,
sometimes it has "version.ref"
sometimes it goes like this "group:Dependency:Version"

TWiStErRob commented

I didn't follow a whole thread yet (I'm on a trip, no access to computer), but are you trying to parse toml with regex? There are many ways to write toml and format code, the version could come first too, it's like a JSON file. Arbitrary order, but also much flexibility on nesting. I'm using version.ref, but it could be ..., version = { ref = "xxx" } } too. Check the Gradle docs.

PhilipAbed commented

no @TWiStErRob i'm trying to find the Location of the Version that im going to change using regex ,
but thats also a good idea,
i could take the toml file and turn it into an object, then change the object version... that's how i would've done it on my own,
but the current design requires me to give the "index" of the location of the version in the EXTRACTION so i can replace it within the UPDATE section later on.

PhilipAbed commented

ok results look good https://github.com/StinkyLord/5838/pull/15/files
i'm creating PR

PhilipAbed commented

hmmm i didn't cover this case:
mylib-full-format = { group = "com.mycompany", name = "alternate", version = { require = "1.4" } }
or
commons-lang3 = { group = "org.apache.commons", name = "commons-lang3", version = { strictly = "[3.8, 4.0[", prefer="3.9" }
or

[versions]
my-lib = { strictly = "[1.0, 2.0[", prefer = "1.2" }

But if regex isnt found then i do a fallback to the old code.

let me check what i can do

PhilipAbed commented

even though these are VALID libs.versions.toml syntax taken from gradle documentation

image
image

the code cant read them properly. it fails to reach the extraction point.
this should have another issue opened for it though, as it's not related to this case.

renovate-release commented

🎉 This issue has been resolved in version 32.48.1 🎉

The release is available on:

Your semantic-release bot 📦🚀