awspring/spring-cloud-aws

Integration with SQS

maciejwalkowiak opened this issue · 3 comments

maciejwalkowiak commented

Integration with SQS must be migrated to AWS SDK v2. This is not meant to be a direct migration, but rather re-thinking how the current one is done. Things to consider:

  • must implement Spring Messaging interfaces
  • must support both regular queues and FIFO queues
  • improve concurrency (#23)
  • think how to extends message visibility when processing FIFO messages (#196)
  • auto extend visibility timeout (spring-attic/spring-cloud-aws#635)
  • support different concurrency configuration per listener

Think about all issues listed under:

tomazfernandes commented

Hi @maciejwalkowiak, would you be willing to look into a community contribution for this, or is it something you're planning on getting done by the core team?

I have some ideas and would be willing to open a draft PR maybe mid next week so we could take a look. Then, of course, if you don't like the design, no worries, should at least bring up some interesting discussions.

What do you think?

Thanks.

  • 👍1
maciejwalkowiak commented

It's been planned to be done by the core team but since we are not getting into it in upcoming weeks you are welcome to contribute! I don't promise it will be merged as it is but hopefully we will be able to work together on it. I don't have yet a design in mind so I am open for ideas.

tomazfernandes commented

Sounds good! I'll open a draft PR with an MVP of what I have in mind then, and if it looks like a good approach we can work our way up from there.

Design-wise, I still have to take a closer look into a few things, but it seems some of the Spring Messaging interfaces, such as the MessageChannel, are not really designed for the kind of async capabilities that the AWS SDK 2.0 allows for. The Handler family seems to have some more interesting options though.

Also, the Spring Kafka project has a solid design on the assembly phase, so we could probably take a few pages from there.

Then there's the flexibility to have a few strategies for polling, acknowledging, and so on, optimizing either for high throughput or low resource usage, or any mix of both.

How does that sound? Please let me know if you have any further inputs or questions, or we can discuss it further in the draft PR.

Thanks!