cal-itp/data-infra

Handle column type change in Airtable `fare_systems` (Fix DbtModelError: model.calitp_warehouse.int_transit_database__fare_systems_dim)

Closed this issue · 0 comments

As a Cal-ITP data user, I want int_transit_database__fare_systems_dim to succeed in the dbt job so that Airtable data is up to date in the warehouse.

Sentry Issue: CAL-ITP-DATA-INFRA-26RD

DbtModelError: model.calitp_warehouse.int_transit_database__fare_systems_dim - Database Error in model int_transit_database__fare_systems_dim (models/intermediate/transit_database/dimensions/int_transit_database__fare_systems_dim.sql)
  Error while reading table: cal-itp-data-infra.external_airtable.california_transit__fare_systems, error message: JSON parsing error in row starting at position 0: Array specified for non-repeated field: reduced_fare_categories. File: gs://calitp-airtable/california_transit__fare_systems/dt=2023-10-26/ts=2023-10-26T02:02:11.599050+00:00/fare_systems.jsonl.gz
  compiled Code at target/run/calitp_warehouse/models/intermediate/transit_database/dimensions/int_transit_database__fare_systems_dim.sql