When sync-diff verifies data, an error is reported if the upstream and downstream tables are inconsistent
jt20190617 opened this issue · 1 comments
Bug Report
Please answer these questions before submitting your issue. Thanks!
- What did you do?
upstream: tidb, and have 2 tables: t1,t2
downstream: tidb, and have 1 tables: t2
Then check the data with sync-diff: ./sync_diff_inspector --config=config.toml
-
What did you expect to see?
Sync-diff notices that there is an additional T1 table upstream, records this information, and proceeds to check the other tables -
What did you see instead?
Sync-diff logs out with an error.
[2022/09/23 17:47:52.913 +08:00] [INFO] [printer.go:46] ["Welcome to sync_diff_inspector"] ["Release Version"=v6.1.1-4-g4572996] ["Git Commit Hash"=45729968487c9b4aa5820b3c00326be0c9bb89d5] ["Git Branch"=master] ["UTC Build Time"="2022-09-23 08:29:09"] ["Go Version"=go1.19.1] [2022/09/23 17:47:52.914 +08:00] [INFO] [main.go:101] [config="{\"check-thread-count\":4,\"split-thread-count\":5,\"export-fix-sql\":true,\"check-struct-only\":false,\"dm-addr\":\"\",\"dm-task\":\"\",\"data-sources\":{\"tidb0\":{\"host\":\"172.16.201.121\",\"port\":4023,\"user\":\"root\",\"password\":\"\",\"sql-mode\":\"\",\"snapshot\":\"\",\"security\":null,\"route-rules\":null,\"Router\":{\"Selector\":{}},\"Conn\":null},\"tidb1\":{\"host\":\"172.16.201.121\",\"port\":4021,\"user\":\"root\",\"password\":\"\",\"sql-mode\":\"\",\"snapshot\":\"\",\"security\":null,\"route-rules\":null,\"Router\":{\"Selector\":{}},\"Conn\":null}},\"routes\":null,\"table-configs\":null,\"task\":{\"source-instances\":[\"tidb1\"],\"source-routes\":null,\"target-instance\":\"tidb0\",\"target-check-tables\":[\"*.*\",\"!INFORMATION_SCHEMA.*\",\"!METRICS_SCHEMA.*\",\"!PERFORMANCE_SCHEMA.*\",\"!mysql.*\",\"!test.*\"],\"target-configs\":null,\"output-dir\":\"./output\",\"SourceInstances\":[{\"host\":\"172.16.201.121\",\"port\":4021,\"user\":\"root\",\"password\":\"\",\"sql-mode\":\"\",\"snapshot\":\"\",\"security\":null,\"route-rules\":null,\"Router\":{\"Selector\":{}},\"Conn\":null}],\"TargetInstance\":{\"host\":\"172.16.201.121\",\"port\":4023,\"user\":\"root\",\"password\":\"\",\"sql-mode\":\"\",\"snapshot\":\"\",\"security\":null,\"route-rules\":null,\"Router\":{\"Selector\":{}},\"Conn\":null},\"TargetTableConfigs\":null,\"TargetCheckTables\":[{},{},{},{},{},{}],\"FixDir\":\"output/fix-on-tidb0\",\"CheckpointDir\":\"output/checkpoint\",\"HashFile\":\"\"},\"ConfigFile\":\"config.toml\",\"PrintVersion\":false}"] [2022/09/23 17:47:53.874 +08:00] [INFO] [tidb.go:199] ["find router for tidb source"] [2022/09/23 17:47:53.874 +08:00] [FATAL] [main.go:120] ["failed to initialize diff process"] [error="from upstream: please make sure the filter is correct.: the target has no table to be compared. source-table is ``db1
.t2``"] [errorVerbose="the target has no table to be compared. source-table is ``db1
.t2``\ngithub.com/pingcap/tidb-tools/sync_diff_inspector/source.checkTableMatched\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/source/source.go:389\ngithub.com/pingcap/tidb-tools/sync_diff_inspector/source.getSourceTableMap\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/source/tidb.go:256\ngithub.com/pingcap/tidb-tools/sync_diff_inspector/source.NewTiDBSource\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/source/tidb.go:263\ngithub.com/pingcap/tidb-tools/sync_diff_inspector/source.buildSourceFromCfg\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/source/source.go:228\ngithub.com/pingcap/tidb-tools/sync_diff_inspector/source.NewSources\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/source/source.go:206\nmain.(*Diff).init\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/diff.go:139\nmain.NewDiff\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/diff.go:97\nmain.checkSyncState\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/main.go:117\nmain.main\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/main.go:104\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:250\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1594\nplease make sure the filter is correct.\nfrom upstream"] [stack="main.checkSyncState\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/main.go:120\nmain.main\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/main.go:104\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:250"]
-
What version of TiDB are you using (
tidb-server -V
or runselect tidb_version();
on TiDB)?
all version -
which tool are you using?
sync-diff -
what versionof tool are you using (
pump -V
ortidb-lightning -V
orsyncer -V
)?
all version
The workaround is set the correct table name with table filter.