gradle/github-dependency-graph-gradle-plugin

Consider excluding `buildSrc` by default.

big-andy-coates opened this issue · 8 comments

From my testing, there seems to be an inconsistency in the way build plugin dependencies are reported, depending on whether they are installed via build scripts under buildSrc vs other build scripts.

Plugins applied in scripts under buildSrc directory are included in the dependency report generated by this project's plugin, including any dependencies the plugins require

vs

Plugins applied in normal build scripts, i.e. not under buildSrc directory, are not included in the dependency report.

The dependencies Gradle task excludes buildSrc by default. Maybe this plugin should also?

Love the work you're doing here!

Andy

For those running into this issue, a workaround when using the gradle-build-action is to set the DEPENDENCY_GRAPH_INCLUDE_PROJECTS env to explicitly exclude buildSrc:

      - name: Setup Gradle
        uses: gradle/gradle-build-action@2
        with:
          dependency-graph: generate
      - name: Build
         env:
           DEPENDENCY_GRAPH_INCLUDE_PROJECTS: "^:(?!buildSrc).*"
         run: ./gradlew build
bigdaz commented

The intent is to include all plugin dependencies in the report, since these can form a critical part of the build infrastructure. I think it's important to be aware if a compromised plugin is being used to build your project.

Plugins applied in normal build scripts, i.e. not under buildSrc directory, are not included in the dependency report.

This is unexpected. Can you provide a reproducer?

Is it possible the difference you're seeing between buildSrc and other plugins is due to filtering for RuntimeClasspath as well? Regular plugin dependencies end up in the classpath configuration of that project, whereas plugins added via buildSrc may end up in the RuntimeClasspath configuration.

bigdaz commented

The GitHub dependency graph model allows dependencies to be flagged as "development" or "runtime", and there are plans to surface this difference in the UI. For example, it should be possible to configure alerts only for "runtime" dependencies. (But at this stage, no such functionality exists).

The Gradle Dependency Graph plugin does not currently categorize dependencies in this way. In fact, it's quite tricky to do so due to the flexibility that Gradle provides when defining dependencies and configurations. But we do plan to add this categorization (via heuristics for standard Java/Android/Spring/etc projects and possibly an optional configuration mapping file).

Once we have this, it should be simpler to say "only include runtime dependencies" rather than having to filter based on projects and configurations. But for now I'd rather leave things as they are, where this is possible, if not very convenient.

The intent is to include all plugin dependencies in the report, since these can form a critical part of the build infrastructure. I think it's important to be aware if a compromised plugin is being used to build your project.

Yes, I can see this as being useful.

I investigated functionality with no filters in a branch, and can confirm it does report dependencies for plugins applied both in buildSrc and in the main build scripts. So all good and consistent on that front and this issue is incorrect.

However, interestingly, the Gradle dependencies task does not report the dependencies of plugins, either applied via buildSrc or the main build scripts, if run normally. It is possible to run (cd buildSrc && ../gradlew dependencies) to have it report plugin dependencies applied in the buildSrc directory, but couldn't find a way to have it include plugins applied in the main build.gradle[.kts] file.

Plugins applied in normal build scripts, i.e. not under buildSrc directory, are not included in the dependency report.

This is unexpected. Can you provide a reproducer?

As above, looks like I was wrong - apologies! - however, it does look like the standard Gradle dependencies task doesn't report plugin dependencies. Or at least I can't work out how to make it do so ;)

So, the description of this issue is wrong about the inconsistency. Maybe it's wrong to exclude plugins by default, but maybe its worth documenting how to quickly limit reported dependencies to runtime, i.e. exclude buildSrc and limit to runtimeClasspath...?

Happy to raise a PR if you want..?

The GitHub dependency graph model allows dependencies to be flagged as "development" or "runtime", and there are plans to surface this difference in the UI. For example, it should be possible to configure alerts only for "runtime" dependencies. (But at this stage, no such functionality exists).

This makes all kinds of sense. I've currently limited the dependencies submitted to "runtimeClasspath" for now, as I only want alerts for security issues that affect downstream users of my project. However, I can see the benefit of reporting all dependencies too, if alerting and the view of dependencies can be filtered (which is outside your control) - otherwise things can get lost in the noise.

The Gradle Dependency Graph plugin does not currently categorize dependencies in this way. In fact, it's quite tricky to do so due to the flexibility that Gradle provides when defining dependencies and configurations. But we do plan to add this categorization (via heuristics for standard Java/Android/Spring/etc projects and possibly an optional configuration mapping file).

Understood. Agree that defaulting to known standard config names, but allowing customisation (via regex?), should give a good balance of "works out of the box" for standard use-cases, but can handle more funky ones too.

Happy to drop in and help if you need.

The intent is to include all plugin dependencies in the report, since these can form a critical part of the build infrastructure. I think it's important to be aware if a compromised plugin is being used to build your project.

This was 100% the original intent of this decision.

The GitHub dependency graph model allows dependencies to be flagged as "development" or "runtime", and there are plans to surface this difference in the UI. For example, it should be possible to configure alerts only for "runtime" dependencies. (But at this stage, no such functionality exists).

One problem is defining what "development" truly means here. In the context of a developer creating a plugin, their "development" dependencies may become "runtime" dependencies. Additionally, compile-only dependencies aren't necessarily "runtime" dependencies, even if they are assumed to be present at runtime by the end product.

bigdaz commented

I investigated functionality with no filters in a branch, and can confirm it does report dependencies for plugins applied both in buildSrc and in the main build scripts. So all good and consistent on that front and this issue is incorrect.

Great, thanks for confirming.

However, interestingly, the Gradle dependencies task does not report the dependencies of plugins, either applied via buildSrc or the main build scripts, if run normally. It is possible to run (cd buildSrc && ../gradlew dependencies) to have it report plugin dependencies applied in the buildSrc directory, but couldn't find a way to have it include plugins applied in the main build.gradle[.kts] file.

You can use the buildEnvironment task to view plugin dependencies.

So, the description of this issue is wrong about the inconsistency. Maybe it's wrong to exclude plugins by default, but maybe its worth documenting how to quickly limit reported dependencies to runtime, i.e. exclude buildSrc and limit to runtimeClasspath...?

Yeah, for now I'm happy with the example in the "Filtering" section, since it excludes buildSrc and restricts to runtimeClasspath. I think that's enough for now.

bigdaz commented

One problem is defining what "development" truly means here. In the context of a developer creating a plugin, their "development" dependencies may become "runtime" dependencies. Additionally, compile-only dependencies aren't necessarily "runtime" dependencies, even if they are assumed to be present at runtime by the end product.

Absolutely. The difficulty in identifying "runtime" vs "development" dependencies (and the lack of any other scopes like "test") is a reason we haven't tackled this yet in the plugin. However, once the GitHub UI starts to make use of these scopes we'll need to come up with some sort of solution (probably using heuristics by default by permitting user configuration).