corrupting disk image

Question

corrupting disk image

wdiechmann opened this issue 6 months ago · 8 comments

ehh - hate to be the guy to cry: wolf!

but I've just spoiled my development.sqlite3 with 10MB of payload - which is not a big deal (but if my production with 50 times that will perform likewise ...)

couldn't help notice the mention on your gem in the plot and as this is my first corrupted DB file since MySQL changed their password strategy back in the day, I'm somewhat edgy 🥹

anything I can do to rule out enhancedsqlite3 being the culprit?

An error occurred when inspecting the object: #<ActiveRecord::StatementInvalid: SQLite3::CorruptException: database disk image is malformed>
An error occurred when running Kernel#inspect: #<ActiveRecord::StatementInvalid: SQLite3::CorruptException: database disk image is malformed>
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/sqlite3-2.0.1-arm64-darwin/lib/sqlite3/statement.rb:36:in `prepare'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/sqlite3-2.0.1-arm64-darwin/lib/sqlite3/statement.rb:36:in `initialize'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/sqlite3-2.0.1-arm64-darwin/lib/sqlite3/database.rb:166:in `new'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/sqlite3-2.0.1-arm64-darwin/lib/sqlite3/database.rb:166:in `prepare'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/activerecord/lib/active_record/connection_adapters/sqlite3/database_statements.rb:36:in `block (2 levels) in internal_exec_query'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/activerecord/lib/active_record/connection_adapters/abstract_adapter.rb:997:in `block in with_raw_connection'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/activesupport/lib/active_support/concurrency/null_lock.rb:9:in `synchronize'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/activerecord/lib/active_record/connection_adapters/abstract_adapter.rb:969:in `with_raw_connection'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/activerecord/lib/active_record/connection_adapters/sqlite3/database_statements.rb:33:in `block in internal_exec_query'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/activesupport/lib/active_support/notifications/instrumenter.rb:58:in `instrument'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/activerecord/lib/active_record/connection_adapters/abstract_adapter.rb:1112:in `log'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/activerecord-enhancedsqlite3-adapter-0.7.0/lib/enhanced_sqlite3/adapter.rb:67:in `block in log'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/activesupport/lib/active_support/tagged_logging.rb:139:in `block in tagged'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/activesupport/lib/active_support/tagged_logging.rb:39:in `tagged'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/activesupport/lib/active_support/tagged_logging.rb:139:in `tagged'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/activesupport/lib/active_support/broadcast_logger.rb:240:in `method_missing'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/activerecord-enhancedsqlite3-adapter-0.7.0/lib/enhanced_sqlite3/adapter.rb:67:in `log'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/activerecord/lib/active_record/connection_adapters/sqlite3/database_statements.rb:32:in `internal_exec_query'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb:114:in `query'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb:110:in `query_values'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/activerecord/lib/active_record/connection_adapters/abstract/schema_statements.rb:35:in `data_sources'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/activerecord/lib/active_record/connection_adapters/schema_cache.rb:430:in `block in tables_to_cache'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb:399:in `with_connection'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/activerecord/lib/active_record/connection_adapters/schema_cache.rb:429:in `tables_to_cache'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/activerecord/lib/active_record/connection_adapters/schema_cache.rb:313:in `data_source_exists?'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/activerecord/lib/active_record/connection_adapters/schema_cache.rb:38:in `data_source_exists?'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/activerecord/lib/active_record/connection_adapters/schema_cache.rb:182:in `data_source_exists?'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/activerecord/lib/active_record/model_schema.rb:418:in `table_exists?'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/activerecord/lib/active_record/core.rb:358:in `inspect'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.13.1/lib/irb/inspector.rb:101:in `inspect'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.13.1/lib/irb/inspector.rb:101:in `bind_call'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.13.1/lib/irb/inspector.rb:101:in `rescue in inspect_value'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.13.1/lib/irb/inspector.rb:95:in `inspect_value'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.13.1/lib/irb/context.rb:639:in `inspect_last_value'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.13.1/lib/irb.rb:1387:in `output_value'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.13.1/lib/irb.rb:1057:in `block (2 levels) in eval_input'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.13.1/lib/irb.rb:1380:in `signal_status'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.13.1/lib/irb.rb:1041:in `block in eval_input'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.13.1/lib/irb.rb:1120:in `block in each_top_level_statement'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.13.1/lib/irb.rb:1117:in `loop'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.13.1/lib/irb.rb:1117:in `each_top_level_statement'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.13.1/lib/irb.rb:1040:in `eval_input'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.13.1/lib/irb.rb:1021:in `block in run'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.13.1/lib/irb.rb:1020:in `catch'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.13.1/lib/irb.rb:1020:in `run'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/railties/lib/rails/commands/console/irb_console.rb:106:in `start'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/railties/lib/rails/commands/console/console_command.rb:59:in `start'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/railties/lib/rails/commands/console/console_command.rb:8:in `start'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/railties/lib/rails/commands/console/console_command.rb:87:in `perform'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/thor-1.3.1/lib/thor/command.rb:28:in `run'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/thor-1.3.1/lib/thor/invocation.rb:127:in `invoke_command'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/railties/lib/rails/command/base.rb:178:in `invoke_command'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/thor-1.3.1/lib/thor.rb:527:in `dispatch'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/railties/lib/rails/command/base.rb:73:in `perform'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/railties/lib/rails/command.rb:71:in `block in invoke'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/railties/lib/rails/command.rb:149:in `with_argv'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/railties/lib/rails/command.rb:69:in `invoke'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/bundler/gems/rails-e3867798a001/railties/lib/rails/commands.rb:18:in `<main>'
<internal:/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:37:in `require'
<internal:/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:37:in `require'
/Users/walther/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bootsnap-1.18.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'

Answer 1 · 2024-05-27T10:48:00.000Z

This is the first I've seen, so I don't have an immediate thought that the gem caused it. We will need to investigate what all you've been doing locally lately, as that stacktrace comes after the corruption occurred. Read thru this documentation page and let me know if anything jumps out as something that might have occurred in your case: https://www.sqlite.org/howtocorrupt.html

Answer 2 · 2024-05-27T11:26:08.000Z

ok - so I see a number of mentions of 'multiple processes' which points me to 'reader' / 'writer' - and in my case I have the database.yml like this

development:
  writer:
    <<: *default
    pool: 1
    database: storage/development.sqlite3
  reader:
    <<: *default
    readonly: true
    database: storage/development.sqlite3

and the solid_queue reading like this:

# config/solid_queue.yml
dispatchers:
  - polling_interval: 1
    batch_size: 500
    recurring_tasks:
      background_job_scheduler:
        interval: 60
        class: BackgroundManagerJob
        queue: default
        schedule: every minute
        enabled: true

and the development configuration like

# config/environments/development.rb
  ...
  config.enhanced_sqlite3.isolate_connection_pools = true
  config.solid_queue.connects_to = { database: { writing: :writer, reading: :reader } }
  ...

This morning I started by loading the app in Chrome - clicking through 4-5 views and then started on my first issue (something time_zone related); finished that, committed the code and then on to #2 which was adding a better root_path => a dashboard. Which meant for me to rails g scaffold dashboard ... and my best guess is that once I started hitting the database after that (successful) migration, things started going south.

I recall rebooting my MBAir this morning after a system upgrade - but I did use the app successfully for at least 10-15min before all was lost.

I am in no way pointing any fingers at you nor others -- only reporting what I see, and hoping by doing so to perhaps keep others from getting "burnt"

cheers,
W

Answer 3 · 2024-05-27T13:05:29.000Z

Are you using a single database for both Active Record and Solid Queue?

Answer 4 · 2024-05-27T13:06:36.000Z

And I take no offense and didn't feel you were pointing fingers. I want to get a better sense of what corrupted your database as well, because we absolutely want to ensure that we don't walk people into corrupt databases. I appreciate the report.

Answer 5 · 2024-05-27T13:53:14.000Z

great!

and - yes - I do use the same DB for both web instance and solid queue

Answer 6 · 2024-05-27T14:01:01.000Z

risked deploying my changes for the day - and they seem to have had no off-setting effect this far.

One of the issues being labeled "SQLite Configuration Errors" is wrong usage of protection mechanisms but unless enhancedsqlite3 chances any of them I don't see it any other way than me having to chalk it up on bad hardware or macOS screwing with the file descriptors - but thanks again for listening and responding so fast!

Answer 7 · 2024-05-27T21:40:14.000Z

You should almost certainly use a separate database for jobs.

Answer 8 · 2024-05-28T05:09:55.000Z

What makes you say that? imho: I was taught ages ago that a 1-file database system is not a proper db - but if 37-signals dare experimenting with SQLite so do I As for the distribution of work, I’m aware that there will be scalability paths to take once the product starts saturating the current VM

…

Den 27. maj 2024 kl. 23.40 skrev Stephen Margheim ***@***.***>: You should almost certainly use a separate database for jobs. — Reply to this email directly, view it on GitHub <#18 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABPFXOENQBR4GEN47QOVMLZEOR5FAVCNFSM6AAAAABIK3QVHGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZUGA3TCNBSGY>. You are receiving this because you modified the open/close state.