Wrappers

Wrappers is a development framework for Postgres Foreign Data Wrappers (FDW), written in Rust. Its goal is to make Postgres FDW development easier while keeping Rust language's modern capabilities, such as high performance, strong types, and safety.

Wrappers is also a collection of FDWs built by Supabase. We currently support the following FDWs, with more are under development:

HelloWorld: A demo FDW to show how to develop a basic FDW.
BigQuery: A FDW for Google BigQuery which supports data read and modify.
Clickhouse: A FDW for ClickHouse which supports data read and modify.
Stripe: A FDW for Stripe API which supports data read and modify.
Firebase: A FDW for Google Firebase which supports data read only.
Airtable: A FDW for Airtable API which supports data read only.
S3: A FDW for AWS S3. Currently read-only.

Features

Minimum interface and easy to implement.
Support for rich data types.
Support both sync and async backends, such as RDBMS, RESTful APIs, flat files and etc.
Built on top of pgrx, providing higher level interfaces, without hiding lower-level C APIs.
WHERE, ORDER BY, LIMIT pushdown are supported.

Documentation

Documentation on docs.rs.

Installation

Wrappers is a pgrx extension, you can follow the pgrx installation steps to install Wrappers.

Basically, run below command to install FDW after pgrx is installed. For example,

cargo pgrx install --pg-config [path_to_pg_config] --features stripe_fdw

Developing a FDW

To develop a FDW using Wrappers, you only need to implement the ForeignDataWrapper trait.

pub trait ForeignDataWrapper {
    // create a FDW instance
    fn new(...) -> Self;

    // functions for data scan, e.g. select
    fn begin_scan(...);
    fn iter_scan(...) -> Option<Row>;
    fn end_scan(...);

    // functions for data modify, e.g. insert, update and delete
    fn begin_modify(...);
    fn insert(...);
    fn update(...);
    fn delete(...);
    fn end_modify(...);

    // other optional functions
    ...
}

In a minimum FDW, which supports data scan only, new(), begin_scan(), iter_scan() and end_scan() are required, all the other functions are optional.

To know more about FDW development, please visit the Wrappers documentation.

Basic usage

These steps outline how to use the a demo FDW HelloWorldFdw, which only outputs a single line of fake data:

Clone this repo

git clone https://github.com/supabase/wrappers.git

Run it using pgrx with feature:

cd wrappers/wrappers
cargo pgrx run pg14 --features helloworld_fdw

Create the extension, foreign data wrapper and related objects:

-- create extension
create extension wrappers;

-- create foreign data wrapper and enable 'HelloWorldFdw'
create foreign data wrapper helloworld_wrapper
  handler hello_world_fdw_handler
  validator hello_world_fdw_validator;

-- create server and specify custom options
create server my_helloworld_server
  foreign data wrapper helloworld_wrapper
  options (
    foo 'bar'
  );

-- create an example foreign table
create foreign table hello (
  id bigint,
  col text
)
  server my_helloworld_server
  options (
    foo 'bar'
  );

Run a query to check if it is working:

wrappers=# select * from hello;
 id |    col
----+-------------
  0 | Hello world
(1 row)

Running tests

In order to run tests in wrappers:

docker-compose -f .ci/docker-compose.yaml up -d
cargo pgrx test --features all_fdws,pg15

Limitations

Windows is not supported, that limitation inherits from pgrx.
Currently only supports PostgreSQL v14 and v15.
Generated column is not supported.

Contribution

All contributions, feature requests, bug report or ideas are welcomed.

License

Apache License Version 2.0

cordada/supabase--wrappers