2.0 Regression: allow_remote_database_url and url_allowlist not working anymore

Question

2.0 Regression: allow_remote_database_url and url_allowlist not working anymore

tmaier opened this issue 4 years ago · 10 comments

Dependabot just upgraded database_cleaner-active_record to 2.0 and now all tests fail.
I used to have DatabaseCleaner.allow_remote_database_url = true in my code.

Open to see error message and trace log

Failure/Error: require File.expand_path('../config/environment', __dir__)

DatabaseCleaner::Safeguard::Error::RemoteDatabaseUrl:
  ENV['DATABASE_URL'] is set to a remote URL. Please refer to https://github.com/DatabaseCleaner/database_cleaner#safeguards
# /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/safeguard.rb:45:in `run'
# /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/safeguard.rb:104:in `block in run'
# /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/safeguard.rb:104:in `each'
# /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/safeguard.rb:104:in `run'
# /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaner.rb:31:in `initialize'
# /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaners.rb:41:in `new'
# /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaners.rb:41:in `add_cleaner'
# /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaners.rb:12:in `block in []'
# /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaners.rb:12:in `fetch'
# /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaners.rb:12:in `[]'
# /usr/local/bundle/gems/database_cleaner-active_record-2.0.0/lib/database_cleaner/active_record.rb:7:in `<main>'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `block in require_with_bootsnap_lfi'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:31:in `require'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:324:in `block in require'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:291:in `load_dependency'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:324:in `require'
# /usr/local/bundle/gems/database_cleaner-active_record-2.0.0/lib/database_cleaner-active_record.rb:1:in `<main>'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `block in require_with_bootsnap_lfi'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:31:in `require'
# /usr/local/bundle/gems/bundler-2.1.4/lib/bundler.rb:174:in `require'
# ./config/application.rb:22:in `<main>'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `block in require_with_bootsnap_lfi'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:31:in `require'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:324:in `block in require'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:291:in `load_dependency'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:324:in `require'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:53:in `require_relative'
# ./config/environment.rb:4:in `<main>'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `block in require_with_bootsnap_lfi'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:31:in `require'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:324:in `block in require'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:291:in `load_dependency'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:324:in `require'
# ./spec/rails_helper.rb:6:in `<main>'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `block in require_with_bootsnap_lfi'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:31:in `require'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:324:in `block in require'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:291:in `load_dependency'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:324:in `require'
# ./spec/uploaders/invoice_uploader_spec.rb:3:in `<main>'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:59:in `load'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:59:in `load'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:318:in `block in load'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:291:in `load_dependency'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:318:in `load'

I also tried to set DatabaseCleaner.url_allowlist = ['postgres://postgres@postgres'], but this did not help.

Answer 1 · 2021-02-01T09:14:28.000Z

Issue opened at wrong repository. Closing it in favour of DatabaseCleaner/database_cleaner-active_record#51

Answer 2 · 2021-02-01T16:26:10.000Z

This is actually the correct place! The safeguard lives in core, so it can be shared by all adapters.

Answer 3 · 2021-02-01T16:34:28.000Z

@tmaier Hmm. The safeguards are still in place, and should be working. I'm guessing its something specific to your usage of database cleaner, plus some refactoring that I did, and together they are having some unintended consequences.

Can you reduce your project down to a minimum reproduction of this issue, and share it with me? If I can reproduce it, I can fix it!

Answer 4 · 2021-02-02T00:41:14.000Z

I had something similar happening here. The error was happening right at the require line, so the code didn't yet had the time to reach the allow_remote_database_url config. Changing to the env var strategy (i.e. exporting DATABASE_CLEANER_ALLOW_REMOTE_DATABASE_URL=true) worked for me.

Answer 5 · 2021-02-02T00:43:18.000Z

Here's the stack:

DatabaseCleaner::Safeguard::Error::RemoteDatabaseUrl:
  ENV['DATABASE_URL'] is set to a remote URL. Please refer to https://github.com/DatabaseCleaner/database_cleaner#safeguards
# /home/xxxxxxxxxxx/.rvm/gems/ruby-2.7.1@xxxxxxxxxxxxxxxxxx/gems/database_cleaner-core-2.0.0/lib/database_cleaner/safeguard.rb:45:in `run'
# /home/xxxxxxxxxxx/.rvm/gems/ruby-2.7.1@xxxxxxxxxxxxxxxxxx/gems/database_cleaner-core-2.0.0/lib/database_cleaner/safeguard.rb:104:in `block in run'
# /home/xxxxxxxxxxx/.rvm/gems/ruby-2.7.1@xxxxxxxxxxxxxxxxxx/gems/database_cleaner-core-2.0.0/lib/database_cleaner/safeguard.rb:104:in `each'
# /home/xxxxxxxxxxx/.rvm/gems/ruby-2.7.1@xxxxxxxxxxxxxxxxxx/gems/database_cleaner-core-2.0.0/lib/database_cleaner/safeguard.rb:104:in `run'
# /home/xxxxxxxxxxx/.rvm/gems/ruby-2.7.1@xxxxxxxxxxxxxxxxxx/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaner.rb:31:in `initialize'
# /home/xxxxxxxxxxx/.rvm/gems/ruby-2.7.1@xxxxxxxxxxxxxxxxxx/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaners.rb:41:in `new'
# /home/xxxxxxxxxxx/.rvm/gems/ruby-2.7.1@xxxxxxxxxxxxxxxxxx/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaners.rb:41:in `add_cleaner'
# /home/xxxxxxxxxxx/.rvm/gems/ruby-2.7.1@xxxxxxxxxxxxxxxxxx/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaners.rb:12:in `block in []'
# /home/xxxxxxxxxxx/.rvm/gems/ruby-2.7.1@xxxxxxxxxxxxxxxxxx/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaners.rb:12:in `fetch'
# /home/xxxxxxxxxxx/.rvm/gems/ruby-2.7.1@xxxxxxxxxxxxxxxxxx/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaners.rb:12:in `[]'
# /home/xxxxxxxxxxx/.rvm/gems/ruby-2.7.1@xxxxxxxxxxxxxxxxxx/gems/database_cleaner-sequel-2.0.0/lib/database_cleaner/sequel.rb:7:in `<top (required)>'
# ./spec/spec_helper.rb:35:in `require'

spec_helper.rb:35 is require "database_cleaner/sequel".

Answer 6 · 2021-02-02T17:57:15.000Z

We're seeing a similar thing happening when we run assets:precompile, because that causes config/application.rb to be loaded, which calls Bundler.require(*Rails.groups) -- simply loading the gem triggers this exception, even if the process never calls it, so there's no way of configuring the DatabaseCleaner class to turn off this safeguard.

Answer 7 · 2021-02-02T21:51:50.000Z

I tried to configure it right after the Bundler.require statement in the application.rb. This did not fix the problem.
So I agree with @jcoglan.

Answer 8 · 2021-02-03T20:52:56.000Z

Okay, opened #673 to attempt to fix this bug. Can you folks try out the branch and let me know if it fixes your issues?

You can do this by changing the database_cleaner line in your Gemfile to the following:

gem "database_cleaner", git: "https://github.com/DatabaseCleaner/database_cleaner", branch: "delay_safeguard"

Answer 9 · 2021-02-03T22:23:17.000Z

Hi @botandrose, I tried it out. It works for me. Thanks mate!‚

Answer 10 · 2021-02-04T16:18:26.000Z

Okay, great! I'll push this out as v2.0.1, presently. Thanks for the detailed reports, detective work, and testing, y'all!