Use metadata as an intermediate format
arnecls opened this issue · 0 comments
arnecls commented
Currently we have a lot of formatters that convert some format to JSON and operate on JSON.
This introduces a lot of Marshal/Unmarshal overhead for longer modulator chains.
As of this we want to remove JSON as the "defacto intermediate format" and instead use Metadata fields to store structured data.
The following formatters should be rebuild and their original version moved to format/deprecated:
- format.GrokToJSON (format.Grok)
- format.TextToJSON (format.Text)
- format.SplitToJSON (format.CSV)
- format.CollectdToInflux* (format.Collectd)
- format.ProcessJSON, format.ProcessTSV, format.ExtractJSON (format.ProcessMetadata)
- format.TemplateJSON (format.TemplateMetadata)
- format.JSONToArray (format.ToArray)
- format.JSONToInflux10 (format.ToInflux10)
- new: format.ToJSON