Does it only support microBatch?
Closed this issue · 5 comments
Not only.
It currently support micro-batch and single batch modes,
that is every mode provisioned by Spark itself, with one exception:
the Continuous Trigger streaming mode is not supported yet.
well, I had used the test-app，but it can't be executed in my idea.Have you test it?
It's a Spark application. You can not execute it directly in idea or anyhow, instead you should submit it into your Spark cluster, then Spark run your application in its distributed environment. Currently we have released only connector for Spark2, so you should install Spark v2.4.8 to be able using it.
You usually submit application into Spark with help of spark-submit utility which come with Spark.
Look the run-test-app.sh scrip in test-app folder for example of submitting application into Spark cluster managed by Yarn cluster manager. For other variants of Spark installation please see Spark documentation here and here.
For the first steps however it is much easier and advisable to run simple tests, like those we shown in our README, interactively via the spark-shell application.
thanks, I will try it
Close as question answered