Example Finagle server that exposes RESTful services to run and manage Apache Spark Jobs.
Requirements:
- JDK 1.8
- gradle
- IntelliJ IDEA (Optional)
- Open the Project with IntellJ, build and run the Main class SparkServerMain.
a) localhost:8080/init - Init the Spark server with any configuration parameter accepted by the Spark Configuration (SparkConf).
Example: curl -D - "localhost:8080/init?spark.master=local&spark.app.name=SparkServer"
b) localhost:8080/shutdown - Shutdown the Spark server and consequently any running spark job.
c) localhost:8080/queryjob - Shows the current status of the Spark Job given by the parameter id
Example: curl -D - localhost:8080/queryjob?id=1
d) localhost:8080/killjob - Kills the Spark Job given by the parameter id
Example: curl -D - localhost:8080/killjob?id=1
e) localhost:8080/runjob - Runs the job specified by the unique identifier id with the particular parameters passed in the url.
Example: curl -D - "localhost:8080/runjob?id=pi&n=100000000"