support for graylog 3

[bjmgeek]

Do you plan to support graylog version 3?

thanks

[nicholas-a-clark]

I got the plugin to at least load by updating <graylog.version> in the pom.xml file to read 3.0.0:

<graylog.plugin-dir>/usr/share/graylog-server/plugin</graylog.plugin-dir>

Then I recompiled the jar from source and put in my plugins directory. I am unsure it is working though or if I need to make any other changes to make it work in 3.0. We use this plugin to injest fastly logs that go into into an S3 bucket.

[cameronattard]

@nicholas-a-clark did that end up working for you?

[nicholas-a-clark]

Yes that worked for me we are using the plugin now

[hmoffatt]

Does anyone have a fork of this plugin that works in 3.3?

[erolugur]

Do you use as (S3->SNS->SQS) or (S3->SQS) ?

I am working on getting ALB Access Logs to Graylog . In this scenario:

1. ALB is configured to put Access Logs on a S3 Bucket
2. Corresponding S3 Bucket is configured to raise an Object Create event to a SQS Queue
3. Using Graylog S3 plugin to read SQS Queue events to get the location of the new log objects on S3 and read those to pass in Input.

In the current implementation plugin:

1. Reads Event from the SQS Queue
2. Tries to parse it as SNS Notification but in my ALB Access Log case, it does not have an notification content so it fails to parse the content and skips the Event

What I am working on plugin to change the behavior like:

1. Read Event from the SQS Queue
2. Parse it as S3 Event (Since S3 directly puts event on SQS without SNS part)
   (From this point kept same)
3. Get the S3 object location from the Event
4. Read Object content and pass each line as input
5. Add an GROK extractor to the input to parse the fields of ALB log line

Am I missing something about using (S3->SQS) instead of (S3->SNS->SQS) ?