jsonnet-libs/k8s

New generator to support CloudFormation resources

Cicatrice opened this issue · 7 comments

Cicatrice commented

Hello,

I work with jsonnet library with more or less 1y now and we are quite happy to manage our k8s resources this way (I work with @xvzf ).

We are on the way to support more resources through jsonnet, and I started a generator for the AWS CloudFormation resources types. It generate yet all the resources types and their properties helpers based on the spec file published by AWS, but obviously, this is a proof-of-concept and it miss a lot of features.

I tried to use most as I can the same coding approach than for original k8s generator, but as I am not generating the same kind of library, there will be a significant drift in model and render (from k8s generator), I started a fork rather than a classic PR.

I wonder if there is a way we can work together on that topic, and maybe have this project adopted in your organisation when it reaches necessary stability

Hope you will find this useful !

xvzf commented

I'm currently thinking about moving this to the jsonnet-libs org though right now it is mostly about k8s - @Duologic @sh0rez any inputs on this? :)

  • 🚀1
Duologic commented

Woha, nice project, the proof-of-concept looks promising already. Looking forward to see it evolve.

We have not discussed about harboring new similar projects, but I'm open to the idea.

Cicatrice commented

Hello,
I moved a bit on that topic ; so far it is supporting all the resources we are using, and we didn't get any trouble. I will try to post a simple demo application in a separate branch of https://github.com/Cicatrice/cfn-libsonnet

The main issue I had is related to the documentation generation (>6 hours) ; I think it is related to the count of files which are generated by this tool (~1k per region per version, there is something like 300k files now).

I choosed to trim the all the regions documentation, and have a default one based on us-east-1 (which is the principal region on AWS). It is now down to ~1h, but it is still quite long (I didnt look yet for caching on mkdocs), I notice you have the same issue with your workflow.

There are some drift between regions spec, that's why I tried to generate all the libraries with region flavours (for exemple, some regions doesnt include features related to the most recent services).

I based this generation on https://github.com/ScriptAutomate/aws-cfn-resource-specs which is community supported (AWS doesn't provide versioned specs files).

xvzf commented

We have a similar issue with the configuration and flakiness of the CI runs, I've been thinking on adding caching for the different libraries and only update them on certain occasions (bazel would be a good candidate here to track dependencies)

I didn't think about mkdocs though. I've never seen anything about builtin caching

Cicatrice commented

hey ! I'm reaching to you about this topic again after some time.

We have used intensively this library now in my company since a year and we me did not have any problem so far with it.

The solution built around it has significantly improve the maintainability of our stacks and projects, and it also accelerate the onboarding of new colleagues.

I noticed jsonnet-libs is now harbouring non-K8s-oriented projects, would you consider this one as well?

Many thanks !

Duologic commented

Would you mind working out a PR to merge changes/additions to the generator rather than hosting a separate generator?

  • 👍1