sqlfluff/sqlfluff

dbt `1.7.14` breaks with `ImportError: cannot import name 'read_user_config' from 'dbt.config'`

Closed this issue · 10 comments

elyobo commented

Search before asking

  • I searched the issues and found no similar issues.

What Happened

dbt-core has just released 1.7.14; installing dbt-core 1.7.14, dbt-bigquery 1.7.7 and the latest sqlfluff 3.0.5 and sqlfluff-templater-dbt 3.0.5 then trying to lint results in ImportError: cannot import name 'read_user_config' from 'dbt.config'

All else the same but using dbt-core 1.7.13 linting works

Probably relevant, dbt-core 1.7.14 release notes - https://github.com/dbt-labs/dbt-core/releases/tag/v1.7.14
dbt-labs/dbt-core#9183 is mentioned and sounds like it might be relevant

Expected Behaviour

linting should work without errors

Observed Behaviour

linting tails with ImportError: cannot import name 'read_user_config' from 'dbt.config'

How to reproduce

install dbt-core 1.7.14, dbt-bigquery 1.7.7, sqlfluff 3.0.5, and sqlfluff-templater-dbt 3.0.5
run sqlfluff lint

Dialect

bigquery

Version

dbt-core 1.7.14, dbt-bigquery 1.7.7, sqlfluff 3.0.5, and sqlfluff-templater-dbt 3.0.5

python --version
Python 3.12.1

poetry run sqlfluff --version
sqlfluff, version 3.0.5

Configuration

probably not relevant

Are you willing to work on and submit a PR to address the issue?

  • Yes I am willing to submit a PR!

Code of Conduct

jtcohen6 commented

@elyobo Thanks for opening the issue!

It would have been this change, which we backported to v1.7.14 + v1.6.14 in order to provide users with a proactive deprecation warning and ability to adopt a more explicit inheritance order for materializations:

In quick summary: dbt.config.project.read_project_flags is replacing dbt.config.profile.read_user_config. Neither of these methods is documented or stable as part of dbt's Python API.

It looks like read_user_config isn't actually being used for dbt >= 1.5, so I think this could be fixed just by moving the import down beneath the else conditional. (Also, dbt-core versions before v1.6 are no longer officially supported.)

sqlfluff/plugins/sqlfluff-templater-dbt/sqlfluff_templater_dbt/templater.py

Lines 148 to 166 in baceed9

from dbt.config import read_user_config
from dbt.config.runtime import RuntimeConfig as DbtRuntimeConfig
# Attempt to silence internal logging at this point.
# https://github.com/sqlfluff/sqlfluff/issues/5054
self.try_silence_dbt_logs()
if self.dbt_version_tuple >= (1, 5):
user_config = None
# 1.5.x+ this is a dict.
cli_vars = self._get_cli_vars()
else:
# Here, we read flags.PROFILE_DIR directly, prior to calling
# set_from_args(). Apparently, set_from_args() sets PROFILES_DIR
# to a lowercase version of the value, and the profile wouldn't be
# found if the directory name contained uppercase letters. This fix
# was suggested and described here:
# https://github.com/sqlfluff/sqlfluff/issues/2253#issuecomment-1018722979
user_config = read_user_config(flags.PROFILES_DIR) 

        from dbt.config.runtime import RuntimeConfig as DbtRuntimeConfig
        ...

        if self.dbt_version_tuple >= (1, 5):
            ...
        else:
            ...      
            from dbt.config import read_user_config
            user_config = read_user_config(flags.PROFILES_DIR)
  • 👍4
anshjain18 commented

I haven't done any version upgrade on either dbt-core or sqlfluff, but still started facing this exact issue suddenly in the past 3-4 days. My versions are:

  1. dbt-core = 1.6.14
  2. sqlfluff = 2.3.0

What can be the possible reasons behind this and how to solve the same?

anshjain18 commented

@greg-finley I think the issue is that the new version where this bug is fixed hasn't been published yet.
The latest version, 3.0.5 was released 2 weeks ago which doesn't seem like it has the fix included

  • 👍3
yingqiaozheng commented

With the dbt-core=1.7.1 the same issue still happening. Anybody has any update? Thank you.

greg-finley commented

Hello @yingqiaozheng did you try sqlfluff 3.0.6 (released last week)? It should be fixed via #5842

derik-roby commented

Unfortunately, I still face this issue with sqlfluff 3.0.6 and dbt-postgres 1.8.

The issue was fixed on downgrading dbt-Postgres to 1.7.14.

elyobo commented

dbt 1.8 introduced other issues, which appear to have been resolved in #5892 but not yet release.

ZZNK commented

stil experiencing this with sqlfluff 3.0.7 and

dbt-core==1.7.10
dbt-postgres==1.7.10
dbt-redshift==1.7.4
dbt-snowflake==1.7.2
dbt-bigquery==1.7.6

  • 👍1
reubster commented

stil experiencing this with sqlfluff 3.0.7 and

dbt-core==1.7.10 dbt-postgres==1.7.10 dbt-redshift==1.7.4 dbt-snowflake==1.7.2 dbt-bigquery==1.7.6

@ZZNK - this was resolved for me by forcing an upgrade of sqlfluff-templater-dbt to 3.0.7 also

  • 👍1
CarMoreno commented

Thanks @reubster , I had to update sqlfluff to 3.0.7 and all works. I am using dbt 1.8.1

dbt-core==1.8.1
dbt-postgres==1.8.1
sqlfluff==3.0.7
sqlfluff-templater-dbt==3.0.7
  • 👍1
  • ❤️1