mallocator/Elasticsearch-Exporter

unable to set number of replicas when exporting form ESCluster to ES Cluster

agastya71 opened this issue · 1 comments

I am trying to export data from one ES cluster to another. (both running ES version 1.5.2)
I cloned the version 2.0 of the Mallocator to help with this. But I get the following when I try to export data with the replicas set to 0 in the destination(Target).
the command I executed :

ode exporter.js -sh source.host.com -sp 7200 -si index1 -th target.host.com -tp 9200 -ti index2 -rc 12 -tr 0 -sq '{"match_all":{}}' -sm 40 -tm 40

and the error I got :
Elasticsearch Exporter - Version 2.0.0
Caught exception in Main process: TypeError: Cannot set property 'number_of_replicas' of undefined
TypeError: Cannot set property 'number_of_replicas' of undefined
at /Volumes/3TBDrive/services/eexporter-git/drivers/elasticsearch.driver.js:424:50
at /Volumes/3TBDrive/services/eexporter-git/node_modules/async/lib/async.js:570:21
at /Volumes/3TBDrive/services/eexporter-git/node_modules/async/lib/async.js:249:17
at /Volumes/3TBDrive/services/eexporter-git/node_modules/async/lib/async.js:125:13
at Array.forEach (native)
at _each (/Volumes/3TBDrive/services/eexporter-git/node_modules/async/lib/async.js:46:24)
at async.each (/Volumes/3TBDrive/services/eexporter-git/node_modules/async/lib/async.js:124:9)
at _asyncMap (/Volumes/3TBDrive/services/eexporter-git/node_modules/async/lib/async.js:248:13)
at Object.map (/Volumes/3TBDrive/services/eexporter-git/node_modules/async/lib/async.js:219:23)
at _parallel (/Volumes/3TBDrive/services/eexporter-git/node_modules/async/lib/async.js:568:20)
Exit code 2: Uncaught Exception

the source had a replication factor of 2 (original and copy). for better index/export performance I thought (as suggested in the README) that perhaps have no replication during the export process and after the exporting is done, to change it.

let me know if anything other details are needed.

Thanks

Ramdev

Thanks for reporting the bug, I just pushed a fix to master.

Btw. some of the options you specified are unnecessary:

-tp 9200: default port is 9200
-sq: default here is also to match all
-rc 12: the elasticsearch driver doesn't support concurrency so it'll be single processed no matter the configuration

I am working on implementing an es driver that supports concurrency, but it's still a while off.

Hope that helps!