ElasticSearch child-parent relationship not work?
tolerious opened this issue · 4 comments
I have two collections, one named books
and the other named bookchapters
, books
is parent of bookchapters
, they both in db engineer
I wrote the config file like this:
var source = mongodb({
"uri": "mongodb://engineer:engineer@localhost:27017/engineer",
})
var sink = elasticsearch({
"uri": "http://localhost:9200/engineer",
"parent_id": "books_id"
})
t.Source("source", source, "/.*/").Save("sink", sink, "/.*/")
without doubt, before I run ./transpoter-0.5.1-darwin-amd64 run pipeline.js
, I have configed the engineer
index like this:
PUT /engineer
{
"mappings":{
"books":{},
"bookchapters":{
"_parent": {
"type": "books"
}
}
}
}
after that I run ./transpoter-0.5.1-darwin-amd64 run pipeline.js
but I got errors like this:
INFO[0002] iterating... collection=bookchapters
ERRO[0002] _bulk failed list [0] error="&elastic.ErrorDetails{Type:"routing_missing_exception", Reason:"routing is required for [engineer]/[bookchapters]/[59e85e58f18d06f0ddc178fa]", ResourceType:"", ResourceId:"", Index:"engineer", Phase:"", Grouped:false, CausedBy:map[string]interface {}(nil), RootCause:[]*elastic.ErrorDetails(nil), FailedShards:[]map[string]interface {}(nil)}" executionID=17 id=59e85e58f18d06f0ddc178fa index=engineer type=bookchapters version=5 writer=elasticsearch
ERRO[0002] _bulk failed list [1] error="&elastic.ErrorDetails{Type:"routing_missing_exception", Reason:"routing is required for [engineer]/[bookchapters]/[59e85e58f18d06f0ddc178fc]", ResourceType:"", ResourceId:"", Index:"engineer", Phase:"", Grouped:false, CausedBy:map[string]interface {}(nil), RootCause:[]*elastic.ErrorDetails(nil), FailedShards:[]map[string]interface {}(nil)}" executionID=17 id=59e85e58f18d06f0ddc178fc index=engineer type=bookchapters version=5 writer=elasticsearch
ERRO[0002] _bulk failed list [2] error="&elastic.ErrorDetails{Type:"routing_missing_exception", Reason:"routing is required for [engineer]/[bookchapters]/[59e85e59f18d06f0ddc178ff]", ResourceType:"", ResourceId:"", Index:"engineer", Phase:"", Grouped:false, CausedBy:map[string]interface {}(nil), RootCause:[]*elastic.ErrorDetails(nil), FailedShards:[]map[string]interface {}(nil)}" executionID=17 id=59e85e59f18d06f0ddc178ff index=engineer type=bookchapters version=5 writer=elasticsearch
ERRO[0002] _bulk failed list [3] error="&elastic.ErrorDetails{Type:"routing_missing_exception", Reason:"routing is required for [engineer]/[bookchapters]/[59e85e5af18d06f0ddc17902]", ResourceType:"", ResourceId:"", Index:"engineer", Phase:"", Grouped:false, CausedBy:map[string]interface {}(nil), RootCause:[]*elastic.ErrorDetails(nil), FailedShards:[]map[string]interface {}(nil)}" executionID=17 id=59e85e5af18d06f0ddc17902 index=engineer type=bookchapters version=5 writer=elasticsearch
ERRO[0002] _bulk failed list [4] error="&elastic.ErrorDetails{Type:"routing_missing_exception", Reason:"routing is required for [engineer]/[bookchapters]/[59e85e5bf18d06f0ddc17903]", ResourceType:"", ResourceId:"", Index:"engineer", Phase:"", Grouped:false, CausedBy:map[string]interface {}(nil), RootCause:[]*elastic.ErrorDetails(nil), FailedShards:[]map[string]interface {}(nil)}" executionID=17 id=59e85e5bf18d06f0ddc17903 index=engineer type=bookchapters version=5 writer=elasticsearch
ERRO[0002] _bulk failed list [5] error="&elastic.ErrorDetails{Type:"routing_missing_exception", Reason:"routing is required for [engineer]/[bookchapters]/[59e85e5df18d06f0ddc17907]", ResourceType:"", ResourceId:"", Index:"engineer", Phase:"", Grouped:false, CausedBy:map[string]interface {}(nil), RootCause:[]*elastic.ErrorDetails(nil), FailedShards:[]map[string]interface {}(nil)}" executionID=17 id=59e85e5df18d06f0ddc17907 index=engineer type=bookchapters version=5 writer=elasticsearch
ERRO[0002] _bulk failed list [6] error="&elastic.ErrorDetails{Type:"routing_missing_exception", Reason:"routing is required for [engineer]/[bookchapters]/[59e85e5df18d06f0ddc17908]", ResourceType:"", ResourceId:"", Index:"engineer", Phase:"", Grouped:false, CausedBy:map[string]interface {}(nil), RootCause:[]*elastic.ErrorDetails(nil), FailedShards:[]map[string]int
What version of Elasticsearch is this?
I'm not super familiar with the parent/child feature in elasticsearch, maybe @johnjjung could comment as he worked on the implementation in transporter?
@codepop @tolerious did you create the parent child relationship before you start the sync?
You must create parent child mappings before you start transporter
Also try running it with the debug flag so you can see more detailed errors
https://github.com/compose/transporter/tree/master/adaptor/elasticsearch#example-of-parent-child-mapping
for example of how to create the mapping on elasticsearch