spark
There are 8212 repositories under spark topic.
apache/spark
Apache Spark - A unified analytics engine for large-scale data processing
donnemartin/data-science-ipython-notebooks
Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.
getredash/redash
Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.
yeasy/docker_practice
Learn and understand Docker&Container technologies, with real DevOps practice!
DataTalksClub/data-engineering-zoomcamp
Free Data Engineering course!
heibaiying/BigData-Notes
大数据入门指南 :star:
GaiZhenbiao/ChuanhuChatGPT
GUI for ChatGPT API and many LLMs. Supports agents, file-based QA, GPT finetuning and query with web search. All with a neat UI.
zhisheng17/flink-learning
flink learning blog. http://www.54tianzhisheng.cn/ 含 Flink 入门、概念、原理、实战、性能调优、源码解析等内容。涉及 Flink Connector、Metrics、Library、DataStream API、Table API & SQL 等内容的学习案例，还有 Flink 落地应用的大型项目案例（PVUV、日志存储、百亿数据实时去重、监控告警）分享。欢迎大家支持我的专栏《大数据实时计算引擎 Flink 实战与性能优化》
horovod/horovod
Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.
aalansehaiyang/technology-talk
【大厂面试专栏】一份Java程序员需要的技术指南，这里有面试题、系统架构、职场锦囊、主流中间件等，让你成为更牛的自己！
FavioVazquez/ds-cheatsheets
List of Data Science Cheatsheets to rule the world
deeplearning4j/deeplearning4j
Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learning using automatic differentiation.
apache/doris
Apache Doris is an easy-to-use, high performance and unified analytics database.
wangzhiwubigdata/God-Of-BigData
专注大数据学习面试，大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive...
mage-ai/mage-ai
🧙 Build, run, and manage data pipelines for integrating and transforming data.
delta-io/delta
An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
h2oai/h2o-3
H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.
Angel-ML/angel
A Flexible and Powerful Parameter Server for large-scale machine learning
Alluxio/alluxio
Alluxio, data orchestration for analytics and machine learning in the cloud
risingwavelabs/risingwave
Cloud-native SQL stream processing, analytics, and management. KsqlDB and Apache Flink alternative. 🚀 10x more productive. 🚀 10x more cost-efficient.
apache/zeppelin
Web-based notebook that enables data-driven, interactive data analytics and collaborative documents with SQL, Scala and more.
donnemartin/dev-setup
macOS development environment setup: Easy-to-understand instructions with automated setup scripts for developer tools like Vim, Sublime Text, Bash, iTerm, Python data analysis, Spark, Hadoop MapReduce, AWS, Heroku, JavaScript web development, Android development, common data stores, and dev-based OS X defaults.
tobymao/sqlglot
Python SQL Parser and Transpiler
microsoft/SynapseML
Simple and Distributed Machine Learning
PipelineAI/pipeline
PipelineAI
yahoo/TensorFlowOnSpark
TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.
Cyb3rWard0g/HELK
The Hunting ELK
JohnSnowLabs/spark-nlp
State of the Art Natural Language Processing
lw-lin/CoolplaySpark
酷玩 Spark: Spark 源代码解析、Spark 类库等
RoaringBitmap/RoaringBitmap
A better compressed bitset in Java: used by Apache Spark, Netflix Atlas, Apache Pinot, Tablesaw, and many others
liyupi/sql-generator
🔨 用 JSON 来生成结构化的 SQL 语句，基于 Vue3 + TypeScript + Vite + Ant Design + MonacoEditor 实现，项目简单（重逻辑轻页面）、适合练手~
databricks/koalas
Koalas: pandas API on Apache Spark
apache/linkis
Apache Linkis builds a computation middleware layer to facilitate connection, governance and orchestration between the upper applications and the underlying data engines.
spark-notebook/spark-notebook
Interactive and Reactive Data Science using Scala and Spark.
awslabs/deequ
Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
WeBankFinTech/DataSphereStudio
DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.