data-infrastructure
There are 77 repositories under data-infrastructure topic.
zalando/postgres-operator
Postgres operator creates and manages PostgreSQL clusters running in Kubernetes
CrunchyData/postgres-operator
Production PostgreSQL for Kubernetes, from high availability Postgres clusters to full-scale database-as-a-service.
zalando/spilo
Highly available elephant herd: HA PostgreSQL cluster using Docker
tensorbase/tensorbase
TensorBase is a new big data warehousing with modern efforts.
zalando/nakadi
A distributed event bus that implements a RESTful API abstraction on top of Kafka-like queues
zalando/PGObserver
A battle-tested, flexible & comprehensive monitoring solution for your PostgreSQL databases
uktrade/stream-unzip
Python function to stream unzip all the files in a ZIP archive on the fly
uktrade/mbtiles-s3-server
Python server to on-the-fly extract and serve vector tiles from an mbtiles file on S3
uktrade/sqlite-s3vfs
Python writable virtual filesystem for SQLite on S3
thedataengineeringbook/thedataengineeringbook
The Data Engineering Book - หนังสือวิศวกรรมข้อมูล ของคนไทย เพื่อคนไทย
uktrade/stream-zip
Python function to construct a ZIP archive on the fly
zalando-incubator/spark-json-schema
JSON schema parser for Apache Spark
abhishek-ch/data-machinelearning-the-boring-way
Build & Learn Data Engineering,Machine Learning over Kubernetes. No Shortcut approach.
uktrade/mobius3
Continuously sync folder to S3, using inotify under the hood
uktrade/fargatespawner
Spawns JupyterHub single user servers in Docker containers running in AWS Fargate
uktrade/data-workspace-frontend
An open source data analysis platform with features for users with a range of technical skills
uktrade/pg-bulk-ingest
Python utility function to ingest data into a SQLAlchemy-defined PostgreSQL table
uktrade/dns-rewrite-proxy
A DNS proxy server that conditionally rewrites and filters A record requests
zalando-nakadi/kanadi
Kanadi is a Nakadi client for Scala
uktrade/stream-sqlite
Python function to extract rows from a SQLite file while iterating over its bytes
uktrade/tidy-json-to-csv
Convert JSON to a set of tidy CSV files
zalando-incubator/darty
Data dependency manager
uktrade/jupyters3
Jupyter Notebook Contents Manager for AWS S3
uktrade/stream-read-xbrl
Python package to parse Companies House accounts data in a streaming way
bizzabo/elasticsearch_to_bigquery_data_pipeline
A generic data pipeline which will map Elasticsearch documents to Bigquery table rows
Jzbonner/dataengineering-db
Information relating to topics on Data Engineering, Data Infrastructure, Data Storing, Data Warehouses and Business Analysis. For those interested in both conceptual theory and use case examples for database design and development.
alphagov/consent-api
Service for sharing user consent to cookies across multiple domains
uktrade/iterable-subprocess
Python context manager to communicate with a subprocess using iterables: for when data is too big to fit in memory and has to be streamed
uktrade/streampq
Python PostgreSQL adapter to stream results of multi-statement queries without a server-side cursor
uktrade/to-file-like-obj
Python utility function to convert an iterable of bytes or str to a readable file-like object
uktrade/jwt-postgresql-proxy
Stateless JWT authentication in front of PostgreSQL
yennanliu/data_infra_repo
Collections of POC/dev data infrastructure. | #SE
anna-geller/kestra-terraform-examples
Bring Infrastructure as Code best practices to your data workflows with Kestra and Terraform
uktrade/public-data-api
The source for the Department for International Trade's Public Data API
uktrade/streamlit-gov-uk-components
A collection of Streamlit components that use or are inspired by the GOV.UK Design System