Catch unsafe migrations at dev time
🍊 Battle-tested at Instacart
Add this line to your application’s Gemfile:
gem 'strong_migrations'
Strong Migrations detects potentially dangerous operations in migrations, prevents them from running by default, and provides instructions on safer ways to do what you want.
__ __ _____ _______ _
\ \ / /\ |_ _|__ __| |
\ \ /\ / / \ | | | | | |
\ \/ \/ / /\ \ | | | | | |
\ /\ / ____ \ _| |_ | | |_|
\/ \/_/ \_\_____| |_| (_) #strong_migrations
ActiveRecord caches attributes which causes problems
when removing columns. Be sure to ignore the column:
class User < ApplicationRecord
self.ignored_columns = %w(some_column)
end
Once that's deployed, wrap this step in a safety_assured { ... } block.
More info: https://github.com/ankane/strong_migrations#removing-a-column
The following operations can cause downtime or errors:
- adding a column with a non-null default value to an existing table
- removing a column
- changing the type of a column
- setting a
NOT NULL
constraint with a default value - renaming a column
- renaming a table
- adding an index non-concurrently (Postgres only)
- adding a
json
column to an existing table (Postgres only)
Also checks for best practices:
- keeping non-unique indexes to three columns or less
Adding a column with a non-null default causes the entire table to be rewritten.
Instead, add the column without a default value, then change the default.
class AddSomeColumnToUsers < ActiveRecord::Migration[5.2]
def up
add_column :users, :some_column, :text
change_column_default :users, :some_column, "default_value"
end
def down
remove_column :users, :some_column
end
end
Don’t backfill existing rows in this migration, as it can cause downtime. See the next section for how to do it safely.
To backfill data, use the Rails console or a separate migration with disable_ddl_transaction!
. Avoid backfilling in a transaction, especially one that alters a table. See this great article on why.
class BackfillSomeColumn < ActiveRecord::Migration[5.2]
disable_ddl_transaction!
def change
# Rails 5+
User.in_batches.update_all some_column: "default_value"
# Rails < 5
User.find_in_batches do |users|
User.where(id: users.map(&:id)).update_all some_column: "default_value"
end
end
end
ActiveRecord caches database columns at runtime, so if you drop a column, it can cause exceptions until your app reboots. To prevent this:
- Tell ActiveRecord to ignore the column from its cache
# For Rails 5+
class User < ApplicationRecord
self.ignored_columns = %w(some_column)
end
# For Rails < 5
class User < ActiveRecord::Base
def self.columns
super.reject { |c| c.name == "some_column" }
end
end
- Deploy code
- Write a migration to remove the column (wrap in
safety_assured
block)
class RemoveSomeColumnFromUsers < ActiveRecord::Migration[5.2]
def change
safety_assured { remove_column :users, :some_column }
end
end
- Deploy and run migration
If you really have to:
- Create a new column
- Write to both columns
- Backfill data from the old column to the new column
- Move reads from the old column to the new column
- Stop writing to the old column
- Drop the old column
One exception is changing a varchar
column to text
, which is safe in Postgres 9.1+.
If you really have to:
- Create a new table
- Write to both tables
- Backfill data from the old table to new table
- Move reads from the old table to the new table
- Stop writing to the old table
- Drop the old table
Add indexes concurrently.
class AddSomeIndexToUsers < ActiveRecord::Migration[5.2]
disable_ddl_transaction!
def change
add_index :users, :some_index, algorithm: :concurrently
end
end
If you forget disable_ddl_transaction!
, the migration will fail. Also, note that indexes on new tables (those created in the same migration) don’t require this.
Check out gindex to quickly generate index migrations without memorizing the syntax.
There’s no equality operator for the json
column type, which causes issues for SELECT DISTINCT
queries.
If you’re on Postgres 9.4+, use jsonb
instead.
If you must use json
, replace all calls to uniq
with a custom scope.
class User < ApplicationRecord
scope :uniq_on_id, -> { select("DISTINCT ON (users.id) users.*") }
end
Then add the column:
class AddJsonColumnToUsers < ActiveRecord::Migration[5.2]
def change
safety_assured { add_column :users, :some_column, :json }
end
end
To mark a step in the migration as safe, despite using method that might otherwise be dangerous, wrap it in a safety_assured
block.
class MySafeMigration < ActiveRecord::Migration[5.2]
def change
safety_assured { remove_column :users, :some_column }
end
end
To mark migrations as safe that were created before installing this gem, create an initializer with:
StrongMigrations.start_after = 20170101000000
Use the version from your latest migration.
For safety, dangerous rake tasks are disabled in production - db:drop
, db:reset
, db:schema:load
, and db:structure:load
. To get around this, use:
SAFETY_ASSURED=1 rake db:drop
Only dump the schema when adding a new migration. If you use Git, create an initializer with:
ActiveRecord::Base.dump_schema_after_migration = Rails.env.development? &&
`git status db/migrate/ --porcelain`.present?
Columns can flip order in db/schema.rb
when you have multiple developers. One way to prevent this is to alphabetize them. Add to the end of your Rakefile
:
task "db:schema:dump": "strong_migrations:alphabetize_columns"
To customize specific messages, create an initializer with:
StrongMigrations.error_messages[:add_column_default] = "Your custom instructions"
Check the source code for the list of keys.
Analyze tables automatically (to update planner statistics) after an index is added. Create an initializer with:
StrongMigrations.auto_analyze = true
It’s a good idea to set a lock timeout for the database user that runs migrations. This way, if migrations can’t acquire a lock in a timely manner, other statements won’t be stuck behind it. Here’s a great explanation of how lock queues work.
ALTER ROLE myuser SET lock_timeout = '10s';
There’s also a gem you can use for this.
Rails 5.1+ uses bigint
for primary keys to keep you from running out of ids. To get this in earlier versions of Rails, check out rails-bigint-primarykey.
Thanks to Bob Remeika and David Waller for the original code.
Everyone is encouraged to help improve this project. Here are a few ways you can help:
- Report bugs
- Fix bugs and submit pull requests
- Write, clarify, or fix documentation
- Suggest or add new features
To get started with development and testing:
git clone https://github.com/ankane/strong_migrations.git
cd strong_migrations
bundle install
bundle exec rake test