[Feature request] Add 'unorderedWaitWithRetry' method to `AsyncDataStream`
Closed this issue · 9 comments
Current version of flink-scala-api seems to not support all featuires of Java's Async I/O API. Implementing unorderedWaitWithRetry would allow to use retry support in asyncronous operations.
As far I can see only unorderedWait and orderedWait are currently implemented.
I would like to tackle this, I'll need retriable async operators implemented too for the migration.
My only question is whether should I reuse the AsyncRetryStrategy from the Java API or create a wrapper like rest of the async interfaces this library implements.
It is new area for me in Flink.
I would follow the existing approach for sake of consistency, however we are open for any meaningful refactoring. Feel free to come up with a proposal to add more operations.
There's one minor detail that makes this difficult, async i/o retry strategy was implemented in 1.16.0 (ref https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=211883963) and this library supports flink 1.15 and upwards. Is there a way to implement features conditionally based on Flink version?
A build for 1.15 version is already dropped from release script: https://github.com/flink-extended/flink-scala-api/blob/master/release.sh
https://github.com/flink-extended/flink-scala-api/blob/master/.github/workflows/ci.yml#L19 does it mean this CI build is outdated? If so, there are no blockers and I'll implement this.
Yes, it needs to be fixed to match the release script.
Great, I've submitted two changes:
- one to deprecate 1.15 in CI builds
- another one to implement async i/o with retries, adds missing spec.
@piotrwieckiewicz I think I'm done and it's only a matter of release.
@buinauskas Thank you very much! Now my terrible Flink code will be just bad with these retries!