fivetran/dbt_zendesk_source

BUG - Freshness checks fail for sources whose tables are excluded from sync

Closed this issue · 1 comments

wuservices commented

Are you a current Fivetran customer?
Jon Wu, Architect, Paper Culture

Describe the bug

When ticket form history isn't being used, freshness is still checked and subsequently causes an error. It should be ignored.

Steps to reproduce

  1. In dbt_project.yml disable form history with using_ticket_form_history: False. In addition, don't select this table for sync in Fivetran so that it doesn't exist.
  2. Run dbt source snapshot-freshness
  3. You'll see an error like ERROR freshness of zendesk.ticket_form_history followed by info saying that the table doesn't exist. For example, something like this for BigQuery:
Runtime Error in source ticket_form_history (models/src_zendesk.yml)
  404 GET https://bigquery.googleapis.com/bigquery/v2/projects/[project]/queries/[query-id]?maxResults=0&location=US&prettyPrint=false: Not found: Table [project]:zendesk.ticket_form_history was not found in location US

Expected behavior

When you set using_ticket_form_history: False, freshness shouldn't be checked. Ideally the source isn't even defined so that it doesn't appear in the docs.

Project variables configuration

name: [project-name]
version: 1.0.0
config-version: 2

profile: default
source-paths: [models]
analysis-paths: [analysis]
test-paths: [tests]
data-paths: [data]
macro-paths: [macros]
snapshot-paths: [snapshots]
target-path: target
clean-targets:
  - target
  - dbt_modules

vars:
  zendesk_source:
    zendesk_database: [project]
  # Disable models for Zendesk features we aren't using or syncing
  using_ticket_form_history: False
  using_schedules: False

Package Version

packages:
  - package: fishtown-analytics/dbt_utils
    version: 0.6.6
  - package: fivetran/ad_reporting
    version: 0.2.2
  - package: fivetran/facebook_ads
    version: 0.2.1
  - package: fivetran/google_ads
    version: 0.2.1
  - package: fivetran/marketo
    version: 0.4.0
  - package: fivetran/microsoft_ads
    version: 0.2.1
  - package: fivetran/pinterest
    version: 0.3.2
  - package: fivetran/zendesk
    version: 0.5.1

Warehouse

  • BigQuery
  • Redshift
  • Snowflake
  • Postgres
  • Databricks
  • Other (provide details below)

Additional context
Similar to fivetran/dbt_hubspot#41

Screenshots

Please indicate the level of urgency

Testing for the first time, but given that this was fixed in another package, it seems like a fix is known.

Are you interested in contributing to this package?

  • Yes, I can do this and open a PR for your review.
  • Possibly, but I'm not quite sure how to do this. I'd be happy to do a live coding session with someone to get this fixed.
  • No, I'd prefer if someone else fixed this. I don't have the time and/or don't know what the root cause of the problem is.
fivetran-jamie commented

included in v0.4.1!

  • 🎉1