[lightdash-gcp] dbt version incompatibility

[rauerhans]

Summary

Lightdash connects to dbt projects and provides CLI commands to generate lightdash-ready model YAMLs, namely
lightdash generate , lightdash deploy , lightdash dbt run.
Currently all of these commands fail on plural deployment of lightdash.
I verified the issue to be a version incompatibility between lightdash and dbt, it's related to

• lightdash/lightdash#4290
• and by extension the true underlying issue https://docs.getdbt.com/guides/migration/versions/upgrading-to-v1.4
  Basically plural's lightdash version 0.3... expects a root_path var in each node in the dbt-generated manifest.json, but dbt >1.4 removed that.

Fix

Bumping the version of the lightdash image to 0.525.0 fixed that.
You would need to bump that version everywhere in the plural app, image, tags, ....

Reproduction

Install any lightdash bundle on a plural cluster and use it with a dbt project using dbt >= 1.4.
The issue is the version incompatibility.

UI/UX Issue Screenshots

For lightdash generate:
(I didn't get a chance to save all errors, because I already fixed it locally by bumping the version in the lightdash image tag)

  ❯ lightdash generate              
(node:63648) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.Please migrate your code to use AWS SDK for JavaScript (v3).For more information, check the migration guide at https://a.co/7PzMCcy
        (Use `node --trace-warnings ...` to show where the warning was created)
        ? Are you sure you want to generate .yml for all models in project? Yes
        Generated .yml files:
        ✖   Failed to generate stg_yfinance__equities_1d.yml
        The "path" argument must be of type string. Received undefined
        TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received undefined
            at new NodeError (node:internal/errors:387:5)
            at validateString (node:internal/validators:162:11)
            at Object.join (node:path:1172:7)
            at /usr/local/lib/node_modules/@lightdash/cli/dist/dbt/models.js:212:49
            at step (/usr/local/lib/node_modules/@lightdash/cli/dist/dbt/models.js:67:23)
            at Object.next (/usr/local/lib/node_modules/@lightdash/cli/dist/dbt/models.js:48:53)
            at /usr/local/lib/node_modules/@lightdash/cli/dist/dbt/models.js:42:71
            at new Promise (<anonymous>)
            at __awaiter (/usr/local/lib/node_modules/@lightdash/cli/dist/dbt/models.js:38:12)
            at findAndUpdateModelYaml (/usr/local/lib/node_modules/@lightdash/cli/dist/dbt/models.js:198:12) {
          code: 'ERR_INVALID_ARG_TYPE'
        }
        TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received undefined
            at new NodeError (node:internal/errors:387:5)
            at validateString (node:internal/validators:162:11)
            at Object.join (node:path:1172:7)
            at /usr/local/lib/node_modules/@lightdash/cli/dist/dbt/models.js:212:49
            at step (/usr/local/lib/node_modules/@lightdash/cli/dist/dbt/models.js:67:23)
            at Object.next (/usr/local/lib/node_modules/@lightdash/cli/dist/dbt/models.js:48:53)
            at /usr/local/lib/node_modules/@lightdash/cli/dist/dbt/models.js:42:71
            at new Promise (<anonymous>)
            at __awaiter (/usr/local/lib/node_modules/@lightdash/cli/dist/dbt/models.js:38:12)
            at findAndUpdateModelYaml (/usr/local/lib/node_modules/@lightdash/cli/dist/dbt/models.js:198:12)
        
        Report this issue with 1-click:
        
          🐛 https://github.com/lightdash/lightdash/issues/new\?assignees\=\&labels\=🐛+bug\&template\=bug_report.md\&title\=The%20%22path%22%20argument%20must%20be%20of%20type%20string.%20Received%20undefined

The error on lightdash dbt run and lightdash deploy was clearer about the root_path, but I don't have that error message anymore.

Additional Info about Your Environment

Tested on google cloud, but should be the same on all clouds.

[davidspek]

Pretty sure this has been fixed already.