elasticsearch-jdbc 插件的使用
v5tech opened this issue · 2 comments
v5tech commented
elasticsearch-1.7.1
elasticsearch-jdbc-1.7.1
https://github.com/jprante/elasticsearch-jdbc
- 插件下载及安装
- 解压及使用
该插件不需要安装,直接解压即可。
注:
使用该插件时需要禁用shield
插件
windows平台下使用时需要修改其示例脚本
@echo off
set DIR=%~dp0
set LIB="%DIR%\..\lib\*"
set BIN="%DIR%\..\bin\*"
REM ???
echo {^
"type" : "jdbc",^
"jdbc" : {^
"url" : "jdbc:mysql://localhost:3306/xiaoboedu",^
"user" : "root",^
"password" : "root",^
"sql" : "SELECT title,subtitle FROM course",^
"autocommit" : true,^
"treat_binary_as_string" : true,^
"elasticsearch" : {^
"cluster" : "elasticsearch",^
"host" : "localhost",^
"port" : 9300^
},^
"index" : "course",^
"type" : "course"^
}^
}^ | "%JAVA_HOME%\bin\java" -cp "%LIB%" -Dlog4j.configurationFile="file://%DIR%\log4j2.xml" "org.xbib.tools.Runner" "org.xbib.tools.JDBCImporter"
其具体使用及配置参见https://github.com/jprante/elasticsearch-jdbc
- 增量更新脚本
@echo off
set DIR=%~dp0
set LIB="%DIR%\..\lib\*"
set BIN="%DIR%\..\bin\*"
REM ???
echo {^
"type" : "jdbc",^
"jdbc" : {^
"url" : "jdbc:mysql://localhost:3306/news",^
"user" : "root",^
"password" : "root",^
"sql" : [^
{"statement":"SELECT title,content,url,source,author,pubdate FROM news"},^
{^
"statement":"SELECT title,content,url,source,author,pubdate FROM news where pubdate > ?",^
"parameter" : [ "$metrics.lastexecutionstart" ]^
}^
],^
"autocommit" : true,^
"treat_binary_as_string" : true,^
"elasticsearch" : {^
"cluster" : "elasticsearch",^
"host" : "localhost",^
"port" : 9300^
},^
"index" : "news",^
"type" : "article"^
}^
}^ | "%JAVA_HOME%\bin\java" -cp "%LIB%" -Dlog4j.configurationFile="file://%DIR%\log4j2.xml" "org.xbib.tools.Runner" "org.xbib.tools.JDBCImporter"
LxiaoGirl commented
请问你在使用这个插件过程中是否出现过以下情况:
我在虚拟机中虚拟了一个window2008的环境,完整使用了该插件,数据导入正常。
我直接拷贝es跟es-jdbc插件到另一台线上windows2008环境中,程序依然能正常执行,但是每次数据数目都不正确,一会多一会少。执行一会就会提示执行完毕,而且没有报错信息。
huweiming commented
我也遇到了同样的问题,全量更新没有问题,增量更新总是少数据