pires/kubernetes-elasticsearch-cluster

Can't access elasticsearch from Kibana

Closed this issue · 5 comments

avitaleli commented

Hi,

When I access Kibana and trying to execute any request to ES (e.g.

POST /my_index/my_book/1
{
  "title": "test book"
})

I get timeout, when checking the logs, I can see an error

[2017-09-10T21:04:21,998][WARN ][r.suppressed ] path: /.kibana/config/_search, params: {index=.kibana, type=config}
org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed

I don't manage to understand, I can see that all my pods are up and running, however, when I manage to execute GET /_cluster/health?pretty
I could see that not all shards are available, but I don't manage to execute this API again.

Any ideas what can be wrong or what I can do to isolate the problem?

Thank you

avitaleli commented

I did some more checks, I can't access elastic search, I get failures.

Executing API http://:9200/_cluster/health?pretty

I get the following:

{
"cluster_name" : "elasticsearch-rtds-dev",
"status" : "yellow",
"timed_out" : false,
"number_of_nodes" : 3,
"number_of_data_nodes" : 1,
"active_primary_shards" : 6,
"active_shards" : 6,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 6,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 50.0
}

For some reason there is a problem with the shards, how do you define your shards?

Thanks

neumachen commented

Any help on this @pires?

Mine is worse:

logstash-2017.10.10 3 p UNASSIGNED    
logstash-2017.10.10 3 r UNASSIGNED    
logstash-2017.10.10 2 p UNASSIGNED    
logstash-2017.10.10 2 r UNASSIGNED    
logstash-2017.10.10 4 p UNASSIGNED    
logstash-2017.10.10 4 r UNASSIGNED    
logstash-2017.10.10 1 p UNASSIGNED    
logstash-2017.10.10 1 r UNASSIGNED    
logstash-2017.10.10 0 p UNASSIGNED    
logstash-2017.10.10 0 r UNASSIGNED    
.kibana             0 p UNASSIGNED    
.kibana             0 r UNASSIGNED
pires commented

@elic9 you only have 1 data-node.

@magicalbanana you're probably having the same issue as elic9. If you don't have enough data-nodes for your replication configuration, you'll have unassigned shards.

neumachen commented

@pires how do I add more data-notes?