S3 access log source fails transformation due to dependency on "name" config field
timoguin opened this issue · 0 comments
timoguin commented
Problem
The managed log source type AWS_S3ACCESS
fails if the name
property in config isn't set to aws_s3access
.
Take this example config:
name: aws_s3_access
managed:
type: AWS_S3ACCESS
ingest:
s3_source:
bucket_name: my-s3-access-logs-bucket
This fails with lots of Failed to find avro schema
errors in the transformer.
It only works if the config is changed to name: aws_s3access
.
Expected Behavior
The name
field should not be depended on for determining data schema in the transformer, since it is a human-facing identifier. These messages make it through the data batcher as expected, and they are tagged appropriately with the aws_s3access
source type.
I expect them to be transformed appropriately.