Use forward input in FARGATE

[LucasHantz]

Hello,

I have 2 containers in my task definition. One being my app in PHP and the other is firelens.
The PHP container is already using the stdout of the container for outputting everything related to the PHP process with basic text output.
My application is reporting logs with some context variables in order at the end to be process through S3/Athena (eg: TraceId, Id related to the app, datetime, log lever, file where the log was fired, etc)
So I'm trying to use a different input by sending JSON to tcp://127.0.0.1:24224 trough the logging lib I use but I don't see anything processed by the firelens container.

So I have 2 questions:

1. Is there a way to do that?
2. Do you have a best approach on how to handle this?

Thanks,
Lucas

[PettitWesley]

Hey @LucasHantz, sorry for the delayed response.

FireLens configures port Fluentd/Bit to listen for Fluent forward messages over tcp port 24224. Raw JSON over TCP won't work; the Fluent forward protocol is different. There is a spec for the fluent forward protocol, but it's not something you'd want to implement yourself. There are libraries in several languages for writing code that talks to Fluentd/Bit using this protocol. For example: https://github.com/fluent/fluent-logger-golang

However, that is probably not the most convenient way to do what you want. If you app is already emitting JSON over TCP, just change the port and use the Fluent Bit TCP input plugin: https://docs.fluentbit.io/manual/input/tcp

Or if you use Fluentd: https://docs.fluentd.org/input/tcp

In Fargate, you unfortunately can not pull a custom config file from S3 (we're working on that), you must embed it the fluentd/bit container image. The FireLens documentation briefly discusses this: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/using_firelens.html

Let me know if you need more help!

[LucasHantz]

I've managed to push the content with the TCP input as you suggested.
Thank you very much for your support!