airbyte-oss

SDK Installation

pip install git+https://github.com/speakeasy-sdks/airbyte-oss-python.git

SDK Example Usage

import airbyte
from airbyte.models import shared

s = airbyte.Airbyte()

req = shared.SaveStatsRequestBody(
    attempt_number=548814,
    job_id=592845,
    stats=shared.AttemptStats(
        bytes_emitted=715190,
        estimated_bytes=844266,
        estimated_records=602763,
        records_committed=857946,
        records_emitted=544883,
        state_messages_emitted=847252,
    ),
    stream_stats=[
        shared.AttemptStreamStats(
            stats=shared.AttemptStats(
                bytes_emitted=623564,
                estimated_bytes=645894,
                estimated_records=384382,
                records_committed=437587,
                records_emitted=297534,
                state_messages_emitted=891773,
            ),
            stream_name='ipsa',
            stream_namespace='delectus',
        ),
        shared.AttemptStreamStats(
            stats=shared.AttemptStats(
                bytes_emitted=272656,
                estimated_bytes=383441,
                estimated_records=477665,
                records_committed=791725,
                records_emitted=812169,
                state_messages_emitted=528895,
            ),
            stream_name='iusto',
            stream_namespace='excepturi',
        ),
    ],
)

res = s.attempt.save_stats(req)

if res.internal_operation_result is not None:
    # handle response

Available Resources and Operations

attempt

save_stats - For worker to set sync stats of a running attempt.
save_sync_config - For worker to save the AttemptSyncConfig for an attempt.
set_workflow_in_attempt - For worker to register the workflow id in attempt.

connection

create_connection - Create a connection between a source and a destination
delete_connection - Delete a connection
get_connection - Get a connection
list_all_connections_for_workspace - Returns all connections for a workspace, including deleted connections.
list_connections_for_workspace - Returns all connections for a workspace.
reset_connection - Reset the data for the connection. Deletes data generated by the connection in the destination. Resets any cursors back to initial state.
search_connections - Search connections
sync_connection - Trigger a manual sync of the connection
update_connection - Update a connection

destination

check_connection_to_destination - Check connection to the destination
check_connection_to_destination_for_update - Check connection for a proposed update to a destination
clone_destination - Clone destination
create_destination - Create a destination
delete_destination - Delete the destination
get_destination - Get configured destination
list_destinations_for_workspace - List configured destinations for a workspace
search_destinations - Search destinations
update_destination - Update a destination

destination_definition

create_custom_destination_definition - Creates a custom destinationDefinition for the given workspace
delete_destination_definition - Delete a destination definition
get_destination_definition - Get destinationDefinition
get_destination_definition_for_workspace - Get a destinationDefinition that is configured for the given workspace
grant_destination_definition_to_workspace - grant a private, non-custom destinationDefinition to a given workspace
list_destination_definitions - List all the destinationDefinitions the current Airbyte deployment is configured to use
list_destination_definitions_for_workspace - List all the destinationDefinitions the given workspace is configured to use
list_latest_destination_definitions - List the latest destinationDefinitions Airbyte supports
list_private_destination_definitions - List all private, non-custom destinationDefinitions, and for each indicate whether the given workspace has a grant for using the definition. Used by admins to view and modify a given workspace's grants.
revoke_destination_definition_from_workspace - revoke a grant to a private, non-custom destinationDefinition from a given workspace
update_destination_definition - Update destinationDefinition

destination_definition_specification

get_destination_definition_specification - Get specification for a destinationDefinition

destination_oauth

complete_destination_o_auth - Given a destination def ID generate an access/refresh token etc.
get_destination_o_auth_consent - Given a destination connector definition ID, return the URL to the consent screen where to redirect the user to.
set_instancewide_destination_oauth_params - Sets instancewide variables to be used for the oauth flow when creating this destination. When set, these variables will be injected into a connector's configuration before any interaction with the connector image itself. This enables running oauth flows with consistent variables e.g: the company's Google Ads developer_token, client_id, and client_secret without the user having to know about these variables.

internal

create_or_update_state - Create or update the state for a connection.
get_attempt_normalization_statuses_for_job - Get normalization status to determine if we can bypass normalization phase
save_stats - For worker to set sync stats of a running attempt.
save_sync_config - For worker to save the AttemptSyncConfig for an attempt.
set_workflow_in_attempt - For worker to register the workflow id in attempt.
write_discover_catalog_result - Should only called from worker, to write result from discover activity back to DB.

jobs

cancel_job - Cancels a job
get_attempt_normalization_statuses_for_job - Get normalization status to determine if we can bypass normalization phase
get_job_debug_info - Gets all information needed to debug this job
get_job_info - Get information about a job
get_job_info_light - Get information about a job excluding attempt info and logs
get_last_replication_job
list_jobs_for - Returns recent jobs for a connection. Jobs are returned in descending order by createdAt.

notifications

try_notification_config - Try sending a notifications

openapi

get_open_api_spec - Returns the openapi specification

operation

check_operation - Check if an operation to be created is valid
create_operation - Create an operation to be applied as part of a connection pipeline
delete_operation - Delete an operation
get_operation - Returns an operation
list_operations_for_connection - Returns all operations for a connection.
update_operation - Update an operation

scheduler

execute_destination_check_connection - Run check connection for a given destination configuration
execute_source_check_connection - Run check connection for a given source configuration
execute_source_discover_schema - Run discover schema for a given source a source configuration

source

check_connection_to_source - Check connection to the source
check_connection_to_source_for_update - Check connection for a proposed update to a source
clone_source - Clone source
create_source - Create a source
delete_source - Delete a source
discover_schema_for_source - Discover the schema catalog of the source
get_most_recent_source_actor_catalog - Get most recent ActorCatalog for source
get_source - Get source
list_sources_for_workspace - List sources for workspace
search_sources - Search sources
update_source - Update a source
write_discover_catalog_result - Should only called from worker, to write result from discover activity back to DB.

source_definition

create_custom_source_definition - Creates a custom sourceDefinition for the given workspace
delete_source_definition - Delete a source definition
get_source_definition - Get source
get_source_definition_for_workspace - Get a sourceDefinition that is configured for the given workspace
grant_source_definition_to_workspace - grant a private, non-custom sourceDefinition to a given workspace
list_latest_source_definitions - List the latest sourceDefinitions Airbyte supports
list_private_source_definitions - List all private, non-custom sourceDefinitions, and for each indicate whether the given workspace has a grant for using the definition. Used by admins to view and modify a given workspace's grants.
list_source_definitions - List all the sourceDefinitions the current Airbyte deployment is configured to use
list_source_definitions_for_workspace - List all the sourceDefinitions the given workspace is configured to use
revoke_source_definition_from_workspace - revoke a grant to a private, non-custom sourceDefinition from a given workspace
update_source_definition - Update a sourceDefinition

source_definition_specification

get_source_definition_specification - Get specification for a SourceDefinition.

source_oauth

complete_source_o_auth - Given a source def ID generate an access/refresh token etc.
get_source_o_auth_consent - Given a source connector definition ID, return the URL to the consent screen where to redirect the user to.
set_instancewide_source_oauth_params - Sets instancewide variables to be used for the oauth flow when creating this source. When set, these variables will be injected into a connector's configuration before any interaction with the connector image itself. This enables running oauth flows with consistent variables e.g: the company's Google Ads developer_token, client_id, and client_secret without the user having to know about these variables.

state

create_or_update_state - Create or update the state for a connection.
get_state - Fetch the current state for a connection.

web_backend

get_state_type - Fetch the current state type for a connection.
web_backend_check_updates - Returns a summary of source and destination definitions that could be updated.
web_backend_create_connection - Create a connection
web_backend_get_connection - Get a connection
web_backend_get_workspace_state - Returns the current state of a workspace
web_backend_list_connections_for_workspace - Returns all non-deleted connections for a workspace.
web_backend_list_geographies - Returns available geographies can be selected to run data syncs in a particular geography. The 'auto' entry indicates that the sync will be automatically assigned to a geography according to the platform default behavior. Entries other than 'auto' are two-letter country codes that follow the ISO 3166-1 alpha-2 standard.
web_backend_update_connection - Update a connection

workspace

create_workspace - Creates a workspace
delete_workspace - Deletes a workspace
get_workspace - Find workspace by ID
get_workspace_by_connection_id - Find workspace by connection id
get_workspace_by_slug - Find workspace by slug
list_workspaces - List all workspaces registered in the current Airbyte deployment
update_workspace - Update workspace state
update_workspace_feedback - Update workspace feedback state
update_workspace_name - Update workspace name

Maturity

This SDK is in beta, and there may be breaking changes between versions without a major version update. Therefore, we recommend pinning usage to a specific package version. This way, you can install the same version each time without breaking changes unless you are intentionally looking for the latest version.

Contributions

While we value open-source contributions to this SDK, this library is generated programmatically. Feel free to open a PR or a Github issue as a proof of concept and we'll do our best to include it in a future release !

yuhuang-cst/airbyte-oss-python