Support for extra inputs
Closed this issue · 5 comments
Right now, prospector.yml (should be renamed to inputs.yml), gets a lot of parameters and expects known inputs like log, stdin, docker and syslog.
There're a lot others like tcp, udp, redis, kafka, ... They would be needed to be implemented one by one, with new parameters and dealing with that huge prospector template file.
Therefore I would like to request a feature to change the inputs to be a pure hash representation for inputs, this way everything would be simpler and future proof when new inputs show up.
This would not be backward compatible and could break existing installs after module upgrade. I'm unsure how you would like to deal with this, probably write some manual conversions on the module, add a deprecate notice on the documentation and remove this conversions on a next major release.
Another possibility is to use a new hash parameter like filebeat::inputs_hash or something else and forget about manual conversions and deprecation, but the old code will probably be not maintained because people will tend to use inputs_hash probably.
I can probably work on that and send a PR, just let me know your toughts.
A purely hash generated method sounds great to me - I've wanted to switch for a while but haven't had time (any PR help is much appreciated). For now some extra flag on the defined resource is probably the easiest and we'll look to making it the default in some future release.
I will implement this, would like your thoughts on best way to do it.
filebeat::inputs is a Hash, filebeat expects an Array of inputs. create_resources() doesn't work on arrays.
My idea is to transform filebeat::inputs into a Variant[Hash, Array] and detects on code if it's a hash follow current implementation otherwise will use pure_hash.yml.erb template.
My only concern with not using the create_resources() function is that it prevents folks from using the defined resource outside of the initial declaration or through hiera.
I wonder if something like a parameter on filebeat::input that accepts a hash that can be converted into a single input might work. When it's present we'd ignore all other options. It'd maintain backwards compatibility and be more future proof, though we'd need some good examples.
Thoughts?
I used create_resources(), check PR lines 147-151. And the extra parameter is there, called pure_array. If pure_array is false (default) backward compatibility is maintained.