kubernetes-csi/livenessprobe

Avoid CVEs with cron! (One simple trick, etc)

briantopping opened this issue · 9 comments

Hello, as mentioned in #135, it seems like this container is (for practical purposes) rarely free of CVEs with a high or critical score. Whether or not they are vulnerable is another story. When referenced artifacts get flagged, it adds significant developer workload to folks who do not know the source of projects like this to dismiss the flags. This is exacerbated when there is a new release and the process of dismissing flags starts anew.

It would be far easier if dependent builds could automatically upgrade to the latest versions of components like this rather than getting involved in deep analysis or improperly dismissing vulnerabilities with a broad (and incorrect) assessment.

I believe the prow configuration for this component is using the golang tag with no bugfix (ie 1.18). A build referencing that tag will pull the latest version of 1.18 on the date of the build.

It follows that this project could be automatically avoid CVEs in older versions of Golang by simply releasing the artifact once a month with an incremented bugfix release. The golang tag will move to the latest version and all will be well. Orgs like ours that use compliance scanning as a blunt force instrument will be satisfied, etc.

Security should be maintained since release branches are used. Merges from master would leverage human review, merges to master from contributions would be ignored unless merged.

@pohly, does this seem sane to you? If it becomes useful, it's a pattern that could be propagated to similar projects as well.

pohly commented

A lot could be done to release the sidecars more frequently. But that's also going to be a lot of work and I don't see anyone volunteering to do that work. If you want to work on this, then we can put this on the agenda for one of the future CSI standup or SIG Storage meetings and discuss what would be required.

I believe the prow configuration for this component is using the golang tag with no bugfix (ie 1.18). A build referencing that tag will pull the latest version of 1.18 on the date of the build.

No, this is v1.18.0. Go only adds .1 when doing the first patch release.

No, this is v1.18.0. Go only adds .1 when doing the first patch release.

$ docker image ls
REPOSITORY                                                                             TAG                   IMAGE ID       CREATED        SIZE
golang                                                                                 1.18                  e7d3e55108dd   2 days ago     820MB
golang                                                                                 1.18.5                e7d3e55108dd   2 days ago     820MB

Did I miss something?

pohly commented

Golang 1.18 is the first release in the 1.18.x release branch: https://go.dev/doc/devel/release#go1.18

We use that semantic in csi-release-tools when checking the compiler version and (if needed) downloading a compiler archive.

That the golang image seems to tag different images with the same 1.18 tag is different and unrelated.

pohly commented

FYI, the release process looks like this:

  • bump Go version in csi-release-tools (manual)
  • propagate to all repos (manual with helper script)
  • test the updates (gap for release branches, would have to be done manually)
  • prepare CHANGELOG (manual)
  • tag release (manual)

All manual steps involve one person making the change and reviewers/approvers accepting it, except for the last one (no reviewer).

If we seriously want to release more often, some of this needs to be automated.

I don't know if Golang is "doing it right" or if this behavior is unintentional and nobody noticed, but the minor tag moves to point to the latest bugfix release. Dependent builds can use the minor tag to get the latest without changing build configuration. It is not a fixed tag. For those who do not want to move to the next minor series without manual intervention, it allows multiple minor release paths to exist and keep a separate latest in each path. It does remove reproducibility as the artifacts generated by a specific git pull are not guaranteed to be identical over time (ie as the tag changes).

Bad assumption on my part that the release process was automated, apologies. I see why this isn't an easy candidate for my suggestion.

I think I counted seven different k8s.io projects with such issues. The problem isn't limited to this project by any means. I guess there's a lot of room for standardization of builds so they are fully automated without creating burden for individual project maintainers.

Anyway, thanks for considering a solution together. :)

pohly commented

I don't know if Golang is "doing it right" or if this behavior is unintentional and nobody noticed, but the minor tag moves to point to the latest bugfix release.

Some images do that. That's useful for images that are used for building things, but also makes image building non-predictable (but that's also the case for other reasons). It's not good practice for images that you deploy in a production cluster because it makes it unpredictable what software runs. For that reason all sidecar images get built once and then never change.

We could fully automate the whole process and "refresh" images automatically by appending a date or number as suffix. But then how do users deploy new sidecars in their cluster when "the latest" sidecar image changes regularly?

We could fully automate the whole process and "refresh" images automatically by appending a date or number as suffix. But then how do users deploy new sidecars in their cluster when "the latest" sidecar image changes regularly?

In case it wasn't clear, I was thinking there was a new release digit or otherwise indicator of change. A date would be fine, or the version of golang could even be appended (which is really all that is changing). I guess it might be awkward if golang didn't change in 30 days, but those days seem to be behind us.

pohly commented

It's not just golang that could have changed. Also any of the third-party dependencies might have a bug fix and thus might have to be updated as part of an image refresh.

Sure, that would be ideal. I was considering what could be done in a purely automated fashion.