snyk/snyk-gradle-plugin

Store gradle & java version meta on every plugin execution

Closed this issue · 10 comments

lili2311 commented

Add more meta on every test run

In order to have better -d output in the CLI for support & to have a general idea of popular version of java & gradle that should be supported it would be good to save this data as meta on the plugin response.

Update the plugin to:

  • run the gradle -v command in the plugin before or after each test run
  • parse the output of it and set it as meta to be returned back with results
  • add a test ito verify parsing is working as expected
  • add tests to assert this meta is being returned by the plugin
  • Snyk team to add the changes in the app UI to render the new meta on the project page

gradle -v

Gradle 4.10.2

Build time:   2018-09-19 18:10:15 UTCRevision:     b4d8d5d170bb4ba516e88d7fe5647e2323d791dd

Kotlin DSL:   1.0-rc-6Kotlin:       1.2.61
Groovy:     2.4.15
Ant:          Apache Ant(TM) version 1.9.11 compiled on March 23 2018
JVM:         10.0.1 ("Oracle Corporation" 10.0.1+10)
OS:           Mac OS X 10.13.2 x86_64

Returned Meta should match:

const pluginMeta = {
  meta: {
    gradle: '4.10.2',
    jvm: '10.0.1',
    os: 'Mac OS X 10.13.2 x86_64',
    groovy: '2.4.15',
  }
}

Please also consider Kotlin, so instead of groovy prop we can expect kotlin?

adrukh commented

Please consider the case where a tested project carries a gradle wrapper gradlew, setting a specific version of gradle for the project.

sfat commented

@lili2311
I've messed around and parsed all the gradle -v output:
I've created an interface containing this information:

{
   "gradleVersion":"4.10.3",
   "metaBuildVersion":[
      {
         "meta":"Build time",
         "value":"2018-12-05 00:50:54 UTC"
      },
      {
         "meta":"Revision",
         "value":"e76905e3a1034e6f724566aeb985621347ff43bc"
      },
      {
         "meta":"Kotlin DSL",
         "value":"1.0-rc-6"
      },
      {
         "meta":"Kotlin",
         "value":"1.2.61"
      },
      {
         "meta":"Groovy",
         "value":"2.4.15"
      },
      {
         "meta":"Ant",
         "value":"Apache Ant(TM) version 1.9.11 compiled on March 23 2018"
      },
      {
         "meta":"JVM",
         "value":"1.8.0_222 (Eclipse OpenJ9 openj9-0.15.1)"
      },
      {
         "meta":"OS",
         "value":"Mac OS X 10.14.6 x86_64"
      }
   ]
}

I think this would cover most of the information required.
Should the gradle plugin also handle the processing of this information or should this be sent as part of PluginMetadata.meta and will be handled down the road?

sfat commented

Or should I ditch the meta and value keys and just make as you presented in the ticket?

sfat commented

this is the ditched meta and value version:

{
   "gradleVersion":"4.10.3",
   "metaBuildVersion":[
      {
         "Build time":"2018-12-05 00:50:54 UTC"
      },
      {
         "Revision":"e76905e3a1034e6f724566aeb985621347ff43bc"
      },
      {
         "Kotlin DSL":"1.0-rc-6"
      },
      {
         "Kotlin":"1.2.61"
      },
      {
         "Groovy":"2.4.15"
      },
      {
         "Ant":"Apache Ant(TM) version 1.9.11 compiled on March 23 2018"
      },
      {
         "JVM":"1.8.0_222 (Eclipse OpenJ9 openj9-0.15.1)"
      },
      {
         "OS":"Mac OS X 10.14.6 x86_64"
      }
   ]
}
sfat commented

@adrukh ,
The gradle plugin itself is picking the wrapper if it finds one in the directory where snyk is running,
I've run some tests against one of the test fixtures that contains a gradle wrapper (test/fixtures/multi-project gradle wrapper)
My local gradle is set to 4.10.3 and this is the output from the build info:

{
   "gradleVersion":"5.4.1",
   "metaBuildVersion":[
      {
         "Build time":"2019-04-26 08:14:42 UTC"
      },
      {
         "Revision":"261d171646b36a6a28d5a19a69676cd098a4c19d"
      },
      {
         "Kotlin":"1.3.21"
      },
      {
         "Groovy":"2.5.4"
      },
      {
         "Ant":"Apache Ant(TM) version 1.9.13 compiled on July 10 2018"
      },
      {
         "JVM":"1.8.0_222 (Eclipse OpenJ9 openj9-0.15.1)"
      },
      {
         "OS":"Mac OS X 10.14.6 x86_64"
      }
   ]
}

Notice that the gradle version is 5.4.1 and is not picking my locally installed gradle.
All in all, I think we should be fine in this regard.

sfat commented

Output from a gradle project with kotlin (test/fixtures/gradle-kts):

{
   "gradleVersion":"5.2.1",
   "metaBuildVersion":[
      {
         "Build time":"2019-02-08 19:00:10 UTC"
      },
      {
         "Revision":"f02764e074c32ee8851a4e1877dd1fea8ffb7183"
      },
      {
         "Kotlin DSL":"1.1.3"
      },
      {
         "Kotlin":"1.3.20"
      },
      {
         "Groovy":"2.5.4"
      },
      {
         "Ant":"Apache Ant(TM) version 1.9.13 compiled on July 10 2018"
      },
      {
         "JVM":"1.8.0_222 (Eclipse OpenJ9 openj9-0.15.1)"
      },
      {
         "OS":"Mac OS X 10.14.6 x86_64"
      }
   ]
}
adrukh commented

I'm not a big fan of arrays of objects, they are not easily searchable in our aggregated logging solution. Also, using camelCase for key strings is also more standard in our code.

Can this just be:

{
   "gradleVersion":"5.2.1",
   "metaBuildVersion": {
         "buildTime":"2019-02-08 19:00:10 UTC",
         "revision":"f02764e074c32ee8851a4e1877dd1fea8ffb7183",
         "kotlinDSL":"1.1.3",
         "kotlin":"1.3.20",
         "groovy":"2.5.4",
         "ant":"Apache Ant(TM) version 1.9.13 compiled on July 10 2018",
         "jvm":"1.8.0_222 (Eclipse OpenJ9 openj9-0.15.1)",
         "os":"Mac OS X 10.14.6 x86_64"
   }
}

?

sfat commented

Hey, @adrukh ,
I've changed it into a {[index: string]: string} instead of array of objects. Looks better, thanks for the tip :)

It looks something like this:

{
   "gradleVersion":"5.2.1",
   "metaBuildVersion":{
      "Build time":"2019-02-08 19:00:10 UTC",
      "Revision":"f02764e074c32ee8851a4e1877dd1fea8ffb7183",
      "Kotlin DSL":"1.1.3",
      "Kotlin":"1.3.20",
      "Groovy":"2.5.4",
      "Ant":"Apache Ant(TM) version 1.9.13 compiled on July 10 2018",
      "JVM":"1.8.0_222 (Eclipse OpenJ9 openj9-0.15.1)",
      "OS":"Mac OS X 10.14.6 x86_64"
   }
}

Regarding the camelCase, I'm all in favor. The thing is that this is what you get from the gradle -v output itself, so haven't gotten that far into modifying it into camelCase.

Do you know if there is a utility library or something that I can use to convert those strings in camelCase or should I implement it myself? Or pick a library that does that?

sfat commented

I've went ahead in not picking a new dependency just for this functionality and did a function instead:

function toCamelCase(input: string) {
  input = input.toLowerCase().replace(/(?:(^.)|([-_\s]+.))/g, (match: string) => {
    return match.charAt(match.length - 1).toUpperCase();
  });
  return input.charAt(0).toLowerCase() + input.substring(1);
}

The output looks as you suggested:

{
   "gradleVersion":"5.2.1",
   "metaBuildVersion":{
      "buildTime":"2019-02-08 19:00:10 UTC",
      "revision":"f02764e074c32ee8851a4e1877dd1fea8ffb7183",
      "kotlinDsl":"1.1.3",
      "kotlin":"1.3.20",
      "groovy":"2.5.4",
      "ant":"Apache Ant(TM) version 1.9.13 compiled on July 10 2018",
      "jvm":"1.8.0_222 (Eclipse OpenJ9 openj9-0.15.1)",
      "os":"Mac OS X 10.14.6 x86_64"
   }
}
  • ❤️1
lili2311 commented 
{
   "gradleVersion":"5.2.1",
   "metaBuildVersion":{
      "buildTime":"2019-02-08 19:00:10 UTC",
      "revision":"f02764e074c32ee8851a4e1877dd1fea8ffb7183",
      "kotlinDsl":"1.1.3",
      "kotlin":"1.3.20",
      "groovy":"2.5.4",
      "ant":"Apache Ant(TM) version 1.9.13 compiled on July 10 2018",
      "jvm":"1.8.0_222 (Eclipse OpenJ9 openj9-0.15.1)",
      "os":"Mac OS X 10.14.6 x86_64"
   }
}

This looks good, we can use this format to display some data on a project page.