/pipeline

PipelineIO: End-to-End ML and AI Platform for Real-time Spark and Tensorflow Data Pipelines

Primary LanguageHTMLApache License 2.0Apache-2.0

PipelineIO

Built with Love and Scale by ex-Netflix, Databricks, and Tensorflow Engineers

Tensorflow Databricks Netflix

Videos

Resources

Documentation

Workshops

Sign Up Now!

Workshop Architecture

Workshop Tools

Quick Start

Local Laptop

Kubernetes Mini + PipelineIO Cluster for your Laptop

  • PipelineIO Local (Local Laptop)

Kubernetes Docker

Kubernetes + PipelineIO Cluster on any Cloud or Data Center

  • Pipeline on AWS

AWS

Google Cloud Platform

Azure

Global 24x7 Support for Everyone!

Reference Architecture

Reference Architecture

Other Features

100% Open Source

Jupyter ZeppelinZeppelin Spark Tensorflow

100% Open Source using Kubernetes + Docker

PipelineIO Kubernetes Kubernetes Docker

PMML + Native C++/Java Code Generation for Optimized CPU/GPU Performance

Generate and Optimize Spark ML Model Nvidia GPU Intel CPU

Deploy directly from Jupyter Notebook to Hybrid-Cloud Microservices

Deploy Spark ML Model to Production

NetflixOSS-based Microservices for High Scalability and Availability

NetflixOSS-based Microservices

Continuous Train/Deploy Spark ML and Tensorflow AI Pipelines to Production

Kafka Kinesis KINESIS

Community

Global Advanced Spark and Tensorflow Meetup

Meetup

Slideshare Slides

Slideshare

YouTube Videos

YouTube

Docker Images

Docker