dbt-ydb

dbt-ydb is a plugin for dbt that provides support for working with YDB. dbt-ydb adapter is in preview stage and does not currently support all dbt features. The sections below list the supported features and known limitations.

Installation

To install plugin, execute the following command:

pip install dbt-ydb

Supported features

Limitations

YDB does not support CTE
YDB requires a primary key to be specified for its tables. See the configuration section for instructions on how to set it.
source() macro requires you to specify a schema. Use / if your source is in root folder.

Usage

Profile Configuration

To configure YDB connection, fill profile.yml file as below:

profile_name:
  target: dev
  outputs:
    dev:
      type: ydb
      host: [localhost] # YDB host
      port: [2136] # YDB port
      database: [/local] # YDB database
      schema: [<empty string>] # Optional subfolder for DBT models
      secure: [False] # If enabled, grpcs protocol will be used
      root_certificates_path: [<empty string>] # Optional path to root certificates file

      # Static Credentials
      username: [<empty string>]
      password: [<empty string>]

      # Access Token Credentials
      token: [<empty string>]

      # Service Account Credentials
      service_account_credentials_file: [<empty string>]

Model Configuration

View

Option	Description	Required	Default

Table

Option	Description	Required	Default
`primary_key`	Primary key expression to use during table creation	`yes`
`store_type`	Type of table. Available options are `row` and `column`	`no`	`row`
`auto_partitioning_by_size`	Enable automatic partitioning by size. Available options are `ENABLED` and `DISABLED`	`no`
`auto_partitioning_partition_size_mb`	Partition size in megabytes for automatic partitioning	`no`
`ttl`	Time-to-live (TTL) expression for automatic data expiration	`no`

Incremental

Option	Description	Required	Default
`incremental_strategy`	Strategy of incremental materialization. Current adapter supports only `merge` strategy, which will use `YDB`'s `UPSERT` operation.	`no`	`default`
`primary_key`	Primary key expression to use during table creation	`yes`
`store_type`	Type of table. Available options are `row` and `column`	`no`	`row`
`auto_partitioning_by_size`	Enable automatic partitioning by size. Available options are `ENABLED` and `DISABLED`	`no`
`auto_partitioning_partition_size_mb`	Partition size in megabytes for automatic partitioning	`no`
`ttl`	Time-to-live (TTL) expression for automatic data expiration	`no`

Example table configuration

{{ config(
    primary_key='id, created_at',
    store_type='row',
    auto_partitioning_by_size='ENABLED',
    auto_partitioning_partition_size_mb=256,
    ttl='Interval("P30D") on created_at'
) }}

select
    id,
    name,
    created_at
from {{ ref('source_table') }}

Seed

Option	Description	Required	Default
`primary_key`	Primary key expression to use during table creation	`no`	The first column of CSV will be used as default.

ydb-platform/dbt-ydb