When locating the database, we need to associate the datatype
FishermanZzhang opened this issue · 1 comments
Versions
eg: 0.7.1
Describe the bug
describe as the title.
For example:
mysql have database test
and clickhouse also have database, which also named test
.
extract metadata mysql fistly and then clickhouse
but clickhouse failed. the ck tables which in database test
, also import to mysql.
To Reproduce
Steps to reproduce the behavior:
- Go to '...'
- Click on '....'
- Scroll down to '....'
- See error
Expected behavior
add databaseType.databasename to locate the database just like the DB which locate the table by databasename.tablename
Actual behavior
If applicable, add screenshots to help explain your problem.
But do not paste log screenshots here.
Full Output Logs
Please paste your log here.
Additional context
Add any other context about the problem here.
关于extract metadata 部分,临时修改了一下代码, 导入成功。
(1)
如果已经存在DBS, 直接修改DBS中唯一键即可。
我使用的mysql
ALTER TABLE DBS drop UNIQUE_DATABASE;
ALTER TABLE DBS add UNIQUE UNIQUE_DATABASE(NAME, DB_TYPE);
如果是在init阶段则需改
https://github.com/Qihoo360/Quicksql/blob/master/metastore/scripts/qsql-metadata-mysql-init.sql#L8
`UNIQUE KEY `UNIQUE_DATABASE` (`NAME`, `DB_TYPE`)`
DatabaseValue origin = client.getBasicDatabaseInfo(dbValue.getName(), dbValue.getDbType());
/**
* select by dbName.
*
* @param databaseName database name
* @return database value
*/
public DatabaseValue getBasicDatabaseInfo(String databaseName) {
return getBasicDatabaseInfo(databaseName, "");
}
/**
* select by dbName.
*
* @param databaseName database name
* @param databaseType database type
* @return database value
*/
public DatabaseValue getBasicDatabaseInfo(String databaseName, String databaseType) {
DatabaseValue databaseValue = null;
String sql = String.format("select DB_ID, `DESC`, NAME, DB_TYPE from DBS where name ='%s'", databaseName);
if (!databaseType.isEmpty()){
sql += String.format("and DB_TYPE='%s'", databaseType);
}
LOGGER.debug("getBasicDatabaseInfo sql is {}", sql);
try (PreparedStatement preparedStatement = connection.prepareStatement(sql)) {
try (ResultSet resultSet = preparedStatement.executeQuery()) {
if (resultSet != null && resultSet.next()) {
databaseValue = new DatabaseValue();
databaseValue.setDbId(resultSet.getLong("DB_ID"));
databaseValue.setName(resultSet.getString("NAME"));
databaseValue.setDbType(resultSet.getString("DB_TYPE"));
databaseValue.setDesc(resultSet.getString("DESC"));
}
}
return databaseValue;
} catch (SQLException ex) {
throw new RuntimeException(ex);
}
}
但查询依然不会成功,原因还是没有加入DB_TYPE
看了一下代码,需要修改fetcher, 如下。
private static class MetadataFetcher {
private String dbType;
private String dbName;
private String tableName;
未完待续。。。