/streamingpro

Build Spark Streaming Application by SQL

Primary LanguageJavaScript

StreamingPro is a fast,expresivive,and convenient cluster system running on Spark with streaming,batch,interactive,machine learning support.

It make devlopers more easy to build spark application without writing any code by means of:

  • Using json file in combination with many powerful modules which are easy to be reused provides the user declarative configurations to build your spark application.
  • Data processing are SQL-Based
  • Script support

StreamingPro is not only a out of box complete application, but also a extensible and programmable framework for spark since you can develop you ower compositors (moduler).

Features

  • Pure Spark Streaming(Or normal Spark) program
  • No need of coding, only declarative workflows
  • Rest API for interactive
  • SQL-Oriented workflows support
  • Data continuously streamed in & processed in near real-time
  • dynamically CURD of workflows at runtime via Rest API
  • Flexible workflows (input, output, parsers, etc...)
  • High performance
  • Scalable

downloads

Precomile version based on Spark 1.6.1 : http://share.weiyun.com/b1d38eb9436d4e05e60b44caee42ce10 http://pan.baidu.com/s/1pKBWa7L

Precomile version based on Spark 2.0.0:  http://share.weiyun.com/62297a8023d6809efa6346370d4b39ea http://pan.baidu.com/s/1mijmgi8

Spark 2.0 based on Scala 2.10 http://pan.baidu.com/s/1mhH1NQS

Documents

Architecture

If no picture show,please click me

If github is too slow to view ,please click me

Declarative workflows for building Spark Streaming

If no picture show,please click me

Implementation

If no picture show,please click me