LuciferYang

Apache Spark Committer&PMC / Apache incubator-uniffle PPMC

@baiduBeijing

Pinned Repositories

3FS
A high-performance distributed file system designed to address the challenges of AI training and inference workloads.
Language:C++00
airflow
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
Language:Python0 0 00
Ammonite
Scala Scripting
Language:Scala0 0 00
arrow
Mirror of Apache Arrow
Language:C++0 2 00
arrow-datafusion-comet
Apache Arrow DataFusion Comet Spark Accelerator
Language:Rust0 0 00
atlasdb
Transactional Distributed Database Layer
Language:Java0 2 00
Awesome-Chinese-LLM
整理开源的中文大语言模型，以规模较小、可私有化部署、训练成本较低的模型为主，包括底座模型，垂直领域微调及应用，数据集与教程等。
0 0 00
bce-qianfan-sdk
Provide best practices for LMOps, as well as elegant and convenient access to the features of the Qianfan MaaS Platform. (提供大模型工具链最佳实践，以及优雅且便捷地访问千帆大模型平台）
Language:Jupyter Notebook00
git
1 1 00
spark
Mirror of Apache Spark
Language:Scala1 3 04

LuciferYang's Repositories

LuciferYang/commons-crypto
Apache Commons Crypto
Language:Java1 0 0
LuciferYang/spark
Mirror of Apache Spark
Language:Scala1 3 04
LuciferYang/3FS
A high-performance distributed file system designed to address the challenges of AI training and inference workloads.
Language:C++00
LuciferYang/Ammonite
Scala Scripting
Language:Scala0 0 00
LuciferYang/arrow
Mirror of Apache Arrow
Language:C++0 2 00
LuciferYang/bce-qianfan-sdk
Provide best practices for LMOps, as well as elegant and convenient access to the features of the Qianfan MaaS Platform. (提供大模型工具链最佳实践，以及优雅且便捷地访问千帆大模型平台）
Language:Jupyter Notebook00
LuciferYang/data-juicer
Making data higher-quality, juicier, and more digestible for foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷为大模型提供更高质量、更丰富、更易”消化“的数据！
LuciferYang/geronimo-xbean
Mirror of Apache Geronimo xbean
Language:Java0 0
LuciferYang/gravitino
A high-performance, geo-distributed and federated metadata lake
Language:Java0 0
LuciferYang/hive
Apache Hive
Language:Java2 0
LuciferYang/iceberg
Apache Iceberg
Language:Java0 0
LuciferYang/incubator-uniffle
Uniffle is a high performance, general purpose Remote Shuffle Service.
Language:Java1 0
LuciferYang/jcasbin
An authorization library that supports access control models like ACL, RBAC, ABAC in Java
Language:Java0 0
LuciferYang/kyuubi
Kyuubi is a unified multi-tenant JDBC interface for large-scale data processing and analytics, built on top of Apache Spark
Language:Scala1 0
LuciferYang/marquez
Collect, aggregate, and visualize a data ecosystem's metadata
LuciferYang/nifi
Apache NiFi
LuciferYang/official-images
Primary source of truth for the Docker "Official Images" program
Language:Shell
LuciferYang/orc
Mirror of Apache Orc
Language:Java2 0
LuciferYang/paimon-trino
Trino Connector for Apache Paimon.
LuciferYang/parquet-mr
Mirror of Apache Parquet
Language:Java2 0
LuciferYang/polaris
Apache Polaris, the interoperable, open source catalog for Apache Iceberg
Language:Java0 0
LuciferYang/ranger
Apache Ranger - To enable, monitor and manage comprehensive data security across the Hadoop platform and beyond
Language:Java
LuciferYang/smallpond
A lightweight data processing framework built on DuckDB and 3FS.
LuciferYang/spark-connect-swift
Apache Spark Connect Client for Swift
LuciferYang/spark-docker
Official Dockerfile for Apache Spark
Language:Shell1 0
LuciferYang/spark-kubernetes-operator
Apache Spark Kubernetes Operator
Language:Java
LuciferYang/spark-upgrade
Magic to help Spark pipelines upgrade
Language:Python1 0
LuciferYang/spark-website
Apache Spark Website
Language:HTML1 0
LuciferYang/unitycatalog
Open, Multi-modal Catalog for Data & AI
Language:Java
LuciferYang/xxHash
Extremely fast non-cryptographic hash algorithm