/incubator-opendal

Apache OpenDAL: Access data freely, painlessly, and efficiently.

Primary LanguageRustApache License 2.0Apache-2.0

OpenDAL   Build Status Latest Version Crate Downloads chat

Open Data Access Layer: Access data freely, painlessly, and efficiently

Services

Features

Access data freely

  • Access different storage services in the same way
  • Behavior tests for all services
  • Cross language/project bindings (working on)

Access data painlessly

Access data efficiently

  • Zero cost: Maps to API calls directly
  • Best effort: Automatically selects best read/seek/next based on services
  • Avoid extra calls: Reuses metadata when possible

Quickstart

use opendal::Result;
use opendal::layers::LoggingLayer;
use opendal::services;
use opendal::Operator;

#[tokio::main]
async fn main() -> Result<()> {
    // Pick a builder and configure it.
    let mut builder = services::S3::default();
    builder.bucket("test");

    // Init an operator
    let op = Operator::create(builder)?
        // Init with logging layer enabled.
        .layer(LoggingLayer::default())
        .finish();

    // Write data
    op.write("hello.txt", "Hello, World!").await?;

    // Read data
    let bs = op.read("hello.txt").await?;

    // Fetch metadata
    let meta = op.stat("hello.txt").await?;
    let mode = meta.mode();
    let length = meta.content_length();

    // Delete
    op.delete("hello.txt").await?;

    Ok(())
}

More examples could be found at Documentation.

Projects

  • Databend: A modern Elasticity and Performance cloud data warehouse.
  • GreptimeDB: An open-source, cloud-native, distributed time-series database.
  • deepeth/mars: The powerful analysis platform to explore and visualize data from blockchain.
  • mozilla/sccache: sccache is ccache with cloud storage
  • risingwave: A Distributed SQL Database for Stream Processing
  • Vector: A high-performance observability data pipeline.

Contributing

Check out the CONTRIBUTING.md guide for more details on getting started with contributing to this project.

Getting help

Submit issues for bug report or asking questions in discussion.

Talk to develops at discord.

License

Licensed under Apache License, Version 2.0.