understanding background migrations

[jjb]

A couple questions about background migrations https://github.com/fatkodima/online_migrations/blob/master/docs/background_migrations.md:

1. why does it have its own entire system, why not just make an ActiveJob and let the normal background job processor do the work?
2. are the phases aware of if the data migration jobs have finished? For example, for https://github.com/fatkodima/online_migrations#changing-the-type-of-a-column, does finalize_column_type_change "know" if the background version of backfill_column_for_type_change has finished yet, and refuse to run if it hasn't?

[fatkodima]

1. not everyone uses ActiveJob, so this is kinda flexibility to be able to be used with sidekiq, cron etc
2. you should manually monitor when its done and then run the next steps

[jjb]

thanks for your responses - a couple questions/thoughts

not everyone uses ActiveJob

You mean not every rails app needs background jobs? Seems to me if someone's app doesn't yet use background jobs, they wouldn't be eager to run cron either, which the (awesome) online_migrations background system depends on.

you should manually monitor when its done and then run the next steps

how to manually monitor this? with the status column in background_migration_jobs?

[fatkodima]

You mean not every rails app needs background jobs?

I mean some people use ActiveJob, some raw Sidekiq. We can create separate job classes for use for ActiveJob, Sidekiq, DelayedJob etc, but there would be still a step for the user to use it somewhere - whether adding to a cron or other scheduler. Not much easier that just for the user to create a simple job manually with 1 line of code calling background migrations and call it in the scheduler of choice.

There is an example for cron in https://github.com/fatkodima/online_migrations/blob/master/docs/background_migrations.md#installation
For ActiveJob this will look something like

class OnlineMigrationsBackgroundJob < ApplicationJob
  def perform
    OnlineMigrations::BackgroundMigrations::Scheduler.run
  end
end

and call it somewhere periodically.

how to manually monitor this? with the status column in background_migration_jobs?

status or progress. See https://github.com/fatkodima/online_migrations/blob/master/docs/background_migrations.md#monitoring-background-migrations