ankane/pgsync

Truncate & defer constraints doesn't play well with partial sync

stem opened this issue · 1 comments

stem commented

Hi again,

I might miss something obvious on this one, but I've got an error where I don't expect one.

Here is the truncated log :

$ bundle exec pgsync --truncate --defer-constraints-v2 --db staging --to-safe partial_sync --debug
From: [...]
To: [...]
[from] SELECT unnest(current_schemas(true)) AS schema
[from] SELECT table_schema AS schema, table_name AS table FROM information_schema.tables WHERE table_type = 'BASE TABLE' AND table_schema NOT IN ('information_schema', 'pg_catalog') ORDER BY 1, 2
[to] SELECT table_schema AS schema, table_name AS table FROM information_schema.tables WHERE table_type = 'BASE TABLE' AND table_schema NOT IN ('information_schema', 'pg_catalog') ORDER BY 1, 2
[from] SELECT table_schema AS schema, table_name AS table, column_name AS column, data_type AS type FROM information_schema.columns WHERE is_generated = 'NEVER' ORDER BY 1, 2, 3
[to] SELECT table_schema AS schema, table_name AS table, column_name AS column, data_type AS type FROM information_schema.columns WHERE is_generated = 'NEVER' ORDER BY 1, 2, 3
[to] SELECT nspname AS schema, relname AS table, pg_attribute.attname AS column, format_type(pg_attribute.atttypid, pg_attribute.atttypmod), pg_attribute.attnum, pg_index.indkey FROM pg_index, pg_class, pg_attribute, pg_namespace WHERE indrelid = pg_class.oid AND pg_class.relnamespace = pg_namespace.oid AND pg_attribute.attrelid = pg_class.oid AND pg_attribute.attnum = any(pg_index.indkey) AND indisprimary
[from] SELECT nt.nspname as schema, t.relname as table, a.attname as column, n.nspname as sequence_schema, s.relname as sequence FROM pg_class s INNER JOIN pg_depend d ON d.objid = s.oid INNER JOIN pg_class t ON d.objid = s.oid AND d.refobjid = t.oid INNER JOIN pg_attribute a ON (d.refobjid, d.refobjsubid) = (a.attrelid, a.attnum) INNER JOIN pg_namespace n ON n.oid = s.relnamespace INNER JOIN pg_namespace nt ON nt.oid = t.relnamespace WHERE s.relkind = 'S'
[to] SELECT nt.nspname as schema, t.relname as table, a.attname as column, n.nspname as sequence_schema, s.relname as sequence FROM pg_class s INNER JOIN pg_depend d ON d.objid = s.oid INNER JOIN pg_class t ON d.objid = s.oid AND d.refobjid = t.oid INNER JOIN pg_attribute a ON (d.refobjid, d.refobjsubid) = (a.attrelid, a.attnum) INNER JOIN pg_namespace n ON n.oid = s.relnamespace INNER JOIN pg_namespace nt ON nt.oid = t.relnamespace WHERE s.relkind = 'S'
shopping_bag_configs: Missing columns: created_at, updated_at
shops: Extra columns: is_asap_deliverable
watermarks: Extra columns: image_fingerprint
[to] BEGIN
[to] SELECT table_schema AS schema, table_name AS table, constraint_name FROM information_schema.table_constraints WHERE constraint_type = 'FOREIGN KEY' AND is_deferrable = 'NO'
[...]
[to] ALTER TABLE "public"."spree_adjustments" ALTER CONSTRAINT "fk_rails_a2ea74df02" DEFERRABLE
[to] ALTER TABLE "public"."spree_adjustments" ALTER CONSTRAINT "fk_spree_adjustments_order_id" DEFERRABLE
[...]
[to] SET CONSTRAINTS ALL DEFERRED
[from] BEGIN
[...]
[to] DELETE FROM "public"."spree_orders"
bundler: failed to load command: pgsync (/Users/stem/.rbenv/versions/2.7.2/lib/ruby/gems/2.7.0/bin/pgsync)
PG::ForeignKeyViolation: ERROR:  update or delete on table "spree_orders" violates foreign key constraint "fk_spree_adjustments_order_id" on table "spree_adjustments"
DETAIL:  Key (id)=(11184) is still referenced from table "spree_adjustments".

As I understand it, it should delete the rows without crashing, sync this table with my custom filter (obviously with some broken foreign keys into spree_adjustments) and then, move on to the next table which will eventually be spree_adjustments where I've setup a filter which match the rows synced in spree_orders
Is that correct ?
Do you have a workaround ?

stem commented

Never mind, I've messed up with the 2 config files and updated the non used one...
Sorry for the noise