amidst/toolbox

How can i extend amidst to accept data from a custom source

srijiths opened this issue · 3 comments

srijiths commented

Is it possible to extend amidst to a custom data source other than file and hdfs ? If so which classes to visit/consider ? It would be grateful if some one point me.

andresmasegosa commented

Yes sure! Anything that implements eu.amidst.core.datastream.DataStream interface can be used to feed the toolbox.
But if you give me more details, I will give a more specific answer.

Thanks for your interests,
AMIDST Team

srijiths commented

@andresmasegosa Thanks for the reply. I was thinking to have a Kafka source with AMIDST. So that i can use AMIDST as part of my stream processing framework ( flink ) . Is it a feasible request ?

andresmasegosa commented

This is indeed a nice request. But we would need to look into that. My first feeling is this is not straightforward. The starting point would be in the flink-link module, in the interface eu.amidst.flinklink.core.data.DataFlink

I have open a new issue for that #80.

Of course, you are welcome to contribute, :).