[Bug] manifest generation error
Opened this issue · 9 comments
Is there an existing issue for this?
- I have searched the existing issues
Describe the issue
I am trying to install the ad reporting model package in fivetran and I have not been able to get it to run.
It runs fine in my local environment, but once I try to run it in fivetran I get the same error.
Details:
- dbt configuration passed all checks
- package: fivetran/ad_reporting
version: 1.7.0
dbt version:
1.6.6
Error:
The following dbt command:
dbt compile
failed during the Manifest Generation process.
The failure details:
Exit code: 2
Output:
07:46:41 Running with dbt=1.6.6
07:46:42 Registered adapter: snowflake=1.6.4
07:46:42 Unable to do partial parsing because saved manifest not found. Starting full parse.
07:47:06 Encountered an error:
Parsing Error
Invalid metrics config given in FilePath(searched_path='models', relative_path='ad_reporting_metrics.yml', modification_time=1674507072.0, project_root='/tmp/dbt/project/dbt_packages/ad_reporting')
The following internal error occurred during the Manifest Generation process:
The 'manifest.json' artifact file not found after executing the command generating the manifest.
Relevant error log or model output
No response
Expected behavior
Deploy ad reporting model
dbt Project configurations
name: 'ad_model'
version: '1.0.0'
config-version: 2
# This setting configures which "profile" dbt uses for this project.
profile: 'ad_model'
# These configurations specify where dbt should look for different types of files.
# The `model-paths` config, for example, states that models in this project can be
# found in the "models/" directory. You probably won't need to change these!
model-paths: ["models"]
analysis-paths: ["analyses"]
test-paths: ["tests"]
seed-paths: ["seeds"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]
clean-targets: # directories to be removed by `dbt clean`
- "target"
- "dbt_packages"
# Configuring models
# Full documentation: https://docs.getdbt.com/docs/configuring-models
# In this example config, we tell dbt to build all models in the example/
# directory as views. These settings can be overridden in the individual model
# files using the `{{ config(...) }}` macro.
models:
ad_model:
# Config indicated by + and applies to all files under models/example/
example:
+materialized: view
vars:
facebook_ads_schema: FACEBOOK_ADS
facebook_ads_database: FIVETRAN_DATABASE
google_ads_schema: GOOGLE_ADS
google_ads_database: FIVETRAN_DATABASE
microsoft_ads_schema: BINGADS
microsoft_ads_database: FIVETRAN_DATABASE
linkedin_ads_schema: LINKEDIN_ADS
linkedin_ads_database: FIVETRAN_DATABASE
twitter_ads_schema: TWITTER_ADS
twitter_ads_database: FIVETRAN_DATABASE
vars:
ad_reporting__amazon_ads_enabled: False # by default this is assumed to be True
ad_reporting__apple_search_ads_enabled: False # by default this is assumed to be True
ad_reporting__pinterest_ads_enabled: False # by default this is assumed to be True
ad_reporting__reddit_ads_enabled: False # by default this is assumed to be True
ad_reporting__snapchat_ads_enabled: False # by default this is assumed to be True
ad_reporting__tiktok_ads_enabled: False # by default this is assumed to be TruePackage versions
package: fivetran/ad_reporting
version: 1.7.0
What database are you using dbt with?
snowflake
dbt Version
1.6.6
Additional Context
No response
Are you willing to open a PR to help address this issue?
- Yes.
- Yes, but I will need assistance and will schedule time during our office hours for guidance
- No.
@rafaelmarinm thanks for opening this issue. We have seen similar issues of customers running into the same manifest error when deploying to either Fivetran or dbt Cloud. Since this runs fine locally, there must be something going on when trying to orchestrate the models and the manifest somehow gets in the way.
@Jstein77 do you have any thoughts as to what this manifest error is? It's interesting that this runs normally in a local environment but not when deploying for orchestration?
Hey @rafaelmarinm sorry for the delay! I was on vacation for the last 2 weeks. I tried to repro this issue using the same dbt version in a clean venv and wasn't able to - the fivetran package installed successfully and dbt compile ran as well. Are you still blocked here?
Hi @rafaelmarinm one quick thought, do you have a package-lock.yml in your project? If so, can you try and delete that file, then rerun dbt clean && dbt deps && dbt run to see if that resolves the issue.
I mainly ask, because @fivetran-catfritz on my team noticed the ad_reporting_metrics file that is causing the issue has since been deprecated following one of the latest releases. I think the old version of ad reporting may still be in use due to the package-lock.yml and we just need to reset it.
Let me know!
@rafaelmarinm Feel free to book some time to go over this issue! https://calendly.com/jordan-stein-1
Hi @rafaelmarinm I am curious if my above suggestion worked for you? Otherwise, were you able to meet with @Jstein77 to investigate further?
I will continue to leave this ticket open in case others have more to share in the thread. However, I will mark the ticket as wontfix since it seems there may not be any updates that need to be applied to the Ad Reporting package.
Please let me know if anyone believes there should be an update to the package. Thanks!
Hi @fivetran-joemarkiewicz, I ran into a similar issue. I'm using dbt version 1.7.3 and tried to run the ad_reporting project on Fivetran. I'm seeing the following error
The node semantic_model.ad_reporting.ad_report not found
It works without any issue if ran locally. Would you be able to help to identify the issue?
As identified within Issue #108, the root cause of this issue seems to be caused by Fivetran transformations not taking into account the semantic_manifest.json. This is not something that will result in an update within the dbt package itself. Instead, if any others come across this issue, please open a Fivetran Support Ticket.
Thank you!