How to run a simple example

[camilomartinez]

I am trying to verify if I have properly setup the framework. Is there any simple example I can run to verify the setup is correct?

I am currently trying with some of the bash scripts but I get an error (e.g. "logparser.sh:: command not found") when running them (e.g. "$(nohup bash logparser.sh &)")

[nemtajo]

There are many already made applications in it.polimi.hyperh.apps and it.polimi.hyperh.experiments packages. A simple application is for example it.polimi.hyperh.apps.LocalApp.scala. When you make your own application you can reuse almost the complete code, with the only modification of cluster deployment mode. Change ".setDeploymentLocalNumExecutors(numOfAlgorithms)" to corresponding deployment mode of your cluster, e.g. ".setDeploymentYarnCluster()".
After you finish writing your application you can run the application locally on your computer by right clicking on the application file->Run as->Scala application.
If you want to try your application on an actual cluster compile the source code and build a jar file using the maven assembly plugin. You can do that by calling Maven build command "package". Right click on the project->Run as->Run configurations, and add a Maven build configuration for which the Goals field has value "package". Then run it.
After the jar has been built (default name is hyperh-0.0.1-SNAPSHOT.jar) copy it to the cluster's file system. In order to run your application you need to invoke a command:
spark-submit --class it.polimi.hyperh.apps.NameOfTheApp hyperh-0.0.1-SNAPSHOT.jar AppArguments. An example:
spark-submit --class it.polimi.hyperh.apps.Experiment1 hyperh-0.0.1-SNAPSHOT.jar 1 8