pingcap/tidb-tools

When sync-diff verifies data, an error is reported if the upstream and downstream tables are inconsistent

jt20190617 opened this issue · 1 comments

Bug Report

Please answer these questions before submitting your issue. Thanks!

  1. What did you do?
    upstream: tidb, and have 2 tables: t1,t2
    downstream: tidb, and have 1 tables: t2

Then check the data with sync-diff: ./sync_diff_inspector --config=config.toml

  1. What did you expect to see?
    Sync-diff notices that there is an additional T1 table upstream, records this information, and proceeds to check the other tables

  2. What did you see instead?
    Sync-diff logs out with an error.
    [2022/09/23 17:47:52.913 +08:00] [INFO] [printer.go:46] ["Welcome to sync_diff_inspector"] ["Release Version"=v6.1.1-4-g4572996] ["Git Commit Hash"=45729968487c9b4aa5820b3c00326be0c9bb89d5] ["Git Branch"=master] ["UTC Build Time"="2022-09-23 08:29:09"] ["Go Version"=go1.19.1] [2022/09/23 17:47:52.914 +08:00] [INFO] [main.go:101] [config="{\"check-thread-count\":4,\"split-thread-count\":5,\"export-fix-sql\":true,\"check-struct-only\":false,\"dm-addr\":\"\",\"dm-task\":\"\",\"data-sources\":{\"tidb0\":{\"host\":\"172.16.201.121\",\"port\":4023,\"user\":\"root\",\"password\":\"\",\"sql-mode\":\"\",\"snapshot\":\"\",\"security\":null,\"route-rules\":null,\"Router\":{\"Selector\":{}},\"Conn\":null},\"tidb1\":{\"host\":\"172.16.201.121\",\"port\":4021,\"user\":\"root\",\"password\":\"\",\"sql-mode\":\"\",\"snapshot\":\"\",\"security\":null,\"route-rules\":null,\"Router\":{\"Selector\":{}},\"Conn\":null}},\"routes\":null,\"table-configs\":null,\"task\":{\"source-instances\":[\"tidb1\"],\"source-routes\":null,\"target-instance\":\"tidb0\",\"target-check-tables\":[\"*.*\",\"!INFORMATION_SCHEMA.*\",\"!METRICS_SCHEMA.*\",\"!PERFORMANCE_SCHEMA.*\",\"!mysql.*\",\"!test.*\"],\"target-configs\":null,\"output-dir\":\"./output\",\"SourceInstances\":[{\"host\":\"172.16.201.121\",\"port\":4021,\"user\":\"root\",\"password\":\"\",\"sql-mode\":\"\",\"snapshot\":\"\",\"security\":null,\"route-rules\":null,\"Router\":{\"Selector\":{}},\"Conn\":null}],\"TargetInstance\":{\"host\":\"172.16.201.121\",\"port\":4023,\"user\":\"root\",\"password\":\"\",\"sql-mode\":\"\",\"snapshot\":\"\",\"security\":null,\"route-rules\":null,\"Router\":{\"Selector\":{}},\"Conn\":null},\"TargetTableConfigs\":null,\"TargetCheckTables\":[{},{},{},{},{},{}],\"FixDir\":\"output/fix-on-tidb0\",\"CheckpointDir\":\"output/checkpoint\",\"HashFile\":\"\"},\"ConfigFile\":\"config.toml\",\"PrintVersion\":false}"] [2022/09/23 17:47:53.874 +08:00] [INFO] [tidb.go:199] ["find router for tidb source"] [2022/09/23 17:47:53.874 +08:00] [FATAL] [main.go:120] ["failed to initialize diff process"] [error="from upstream: please make sure the filter is correct.: the target has no table to be compared. source-table is ``db1.t2``"] [errorVerbose="the target has no table to be compared. source-table is ``db1.t2``\ngithub.com/pingcap/tidb-tools/sync_diff_inspector/source.checkTableMatched\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/source/source.go:389\ngithub.com/pingcap/tidb-tools/sync_diff_inspector/source.getSourceTableMap\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/source/tidb.go:256\ngithub.com/pingcap/tidb-tools/sync_diff_inspector/source.NewTiDBSource\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/source/tidb.go:263\ngithub.com/pingcap/tidb-tools/sync_diff_inspector/source.buildSourceFromCfg\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/source/source.go:228\ngithub.com/pingcap/tidb-tools/sync_diff_inspector/source.NewSources\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/source/source.go:206\nmain.(*Diff).init\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/diff.go:139\nmain.NewDiff\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/diff.go:97\nmain.checkSyncState\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/main.go:117\nmain.main\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/main.go:104\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:250\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1594\nplease make sure the filter is correct.\nfrom upstream"] [stack="main.checkSyncState\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/main.go:120\nmain.main\n\t/home/jenkins/agent/workspace/d_tidb_tools_multi_branch_master/go/src/github.com/pingcap/tidb-tools/sync_diff_inspector/main.go:104\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:250"]

  3. What version of TiDB are you using (tidb-server -V or run select tidb_version(); on TiDB)?
    all version

  4. which tool are you using?
    sync-diff

  5. what versionof tool are you using (pump -V or tidb-lightning -V or syncer -V)?
    all version

The workaround is set the correct table name with table filter.