InsightEdge Examples
InsightEdge GigaSpaces convergence platform
Introduction
InsightEdge is a Hybrid transactional/analytical processing platform built on top of Spark and GigaSpaces Data Grid.
This project contains examples of how standard Spark applications can use Data Grid for different purposes.
Requirements
- Java 1.8
- Scala 2.10
- Maven 3.1+
- SBT 0.13 (optional)
- InsightEdge distribution
Building project
InsightEdge jars are not published to Maven Central Repository yet. To install artifacts to your local Maven repository, make sure you have Maven installed and then run:
./sbin/insightedge-maven.sh
This project has both SBT and Maven configurations. You can build it with next commands:
# Maven
mvn clean test package
# SBT
sbt clean test assembly
Running examples
There are several options how you can run examples:
- from Web Notebook (refer to corresponding section in InsightEdge documentation)
- from your IDE
- from a command line
Starting local environment
Prior to executing example application, you have to start the Data Grid and deploy an empty space on it. You can do it using demo
mode:
./sbin/insightedge.sh --mode demo
Such command will start next components:
- Spark master at
spark://127.0.0.1:7077
and Spark slave - Data Grid manager and two containers with
1G
heap each- space is deployed with name
insightedge-space
- lookup locator is
127.0.0.1:4174
- lookup group is
insightedge
- space is deployed with name
Running from IDE
You can run examples from your favourite IDE. Every example has a main
method, so it can be executed as standard application.
Here is an example of run configuration for SaveRDD
for Intellij Idea
:
With this configuration, example will run on local Spark cluster and save the generated RDD to Data Grid to specified space.
Running from command line
You can build the project and submit examples as Spark applications with the next command:
./bin/insightedge-submit --class {main class name} --master {Spark master URL} \
{insightedge-examples.jar location} \
{Spark master URL} {space name} {lookup group} {lookup locator}
For example, SaveRDD
can be submitted with the next syntax:
./bin/insightedge-submit --class com.gigaspaces.insightedge.examples.basic.SaveRdd --master spark://127.0.0.1:7077 \
/home/user/Git/insightedge-examples/target/insightedge-examples.jar \
spark://127.0.0.1:7077 insightedge-space insightedge 127.0.0.1:4174
Note that running
TwitterPopularTags
example requires you to pass Twitter app tokens as arguments
Stopping local environment
To stop all InsightEdge components, next command can be executed:
./sbin/insightedge.sh --mode shutdown
Troubleshooting
If you have any troubles running the example applications, please, contact us with:
- Slack channel using invitation
- StackOverflow insightedge tag
- contact form at InsightEdge main page
- or email message