split pipecleaner to its own repository

Question

split pipecleaner to its own repository

afeld opened this issue 8 years ago · 14 comments

https://github.com/alphagov/paas-cf/blob/master/concourse/scripts/pipecleaner.py

@jmcarp just pointed me to this script—seems super useful! Would you consider breaking it out to its own repository, so it can be installed/run more easily?

Answer 1 · 2016-10-05T04:27:28.000Z

Heh, sorry, can see you're ahead of me: vmware-archive/fly#99

Answer 2 · 2016-10-05T10:23:45.000Z

@afeld I think that seems sensible, I'll talk to the team and see what we can do!

Answer 3 · 2016-10-14T10:22:09.000Z

@afeld We agree this seems like a good idea, sorry we took a while to get back to you, but we have now scheduled this for our next iteration

Answer 4 · 2016-10-14T10:36:00.000Z

FYI, next iteration starts Wednesday 19th October :)

Answer 5 · 2016-10-14T15:56:21.000Z

Once I get this PR against fly in good shape, there will be a fly validate-pipeline command that hopefully does much of what pipecleaner does now, although it obviously won't include shellcheck.

Answer 6 · 2016-10-24T14:16:41.000Z

Ok, we've pushed it down in the priority list, since we'll likely want to use fly validate-pipeline ourselves.

If it's useful to anyone that we go ahead and split out pipecleaner into its own repo, please say so.

Answer 7 · 2017-01-07T06:27:23.000Z

The fly cli learned validate-pipeline in concourse 2.6. Does pipecleaner do validation that concourse doesn't? If not, would be great to file issues / patches upstream!

Answer 8 · 2017-11-02T13:37:25.000Z

@jmcarp just been reminded of this.

There's a description of what pipecleaner does - it's a bit more of a linter than a validator.

It runs shellcheck against the shell commands in the pipeline and checks for unused resources etc.

I think validate-pipeline mostly checks that the YAML will parse ok and doesn't do these logical checks. That's probably OK in most cases as shellcheck is relatively opinionated.

Answer 9 · 2017-11-02T13:40:29.000Z

@bleach Pipecleaner also validates the dependency tree between steps/jobs, which I'm not sure validate-pipeline does either.

Answer 10 · 2017-11-02T13:52:47.000Z

@Jonty agreed. I probably wasn't clear enough that the comment in pipecleaner I linked to is a better description of what it does :)

Answer 11 · 2017-11-02T13:57:53.000Z

@bleach Pipecleaner also validates the dependency tree between steps/jobs, which I'm not sure validate-pipeline does either.

I think it does. Hi, btw!

Answer 12 · 2017-11-03T01:32:41.000Z

@bleach: fly validate-pipeline doesn't do everything that pipecleaner does, but it looks like there's some overlap. A few things validate-pipeline checks that also happen in pipecleaner:

A job uses a resource that isn't defined: https://github.com/concourse/atc/blob/master/validate.go#L414-L424
A resource is defined but not used in any jobs: https://github.com/concourse/atc/blob/master/validate.go#L164-L176

Answer 13 · 2018-01-19T09:10:44.000Z

@afeld we were wondering if you were still interested in this, or if fly validate-pipeline meets your needs?

Answer 14 · 2018-01-24T05:09:15.000Z

@bleach I haven't been using Concourse much recently (moved projects), but sounds like y'all are off and running with this. @jmcarp is still on that team, so he is a better judge than me. Will close this issue, since it sounds like the remaining work could be migrating more of the pipecleaner functionality into fly validate-pipeline. Thanks everyone!