datafari 4.2 fails to initialize ELK

Question

datafari 4.2 fails to initialize ELK

marlemion opened this issue 6 years ago · 1 comments

After installation of datafari 4.2, a clean first start does not initialize ELK (and, hence, installation fails).

Initializing ELK...
Set logs paths...
Logs paths correctly set
Initialize Logstash...
Logstash initialized !
Initialize Kibana...
Kibana initialized !
Warn: a PID file was detected, removing it.
Starting Elasticsearch...
Elasticsearch started !
Checking if Elasticsearch is up and running ...
/opt/datafari/elk/scripts/elk-utils.sh: connect: Connection refused
/opt/datafari/elk/scripts/elk-utils.sh: line 59: /dev/tcp/localhost/9200: Connection refused
Elasticsearch doesn't reply to requests on port 9200. Sleeping for a while and trying again... retry 1
Warning: Ignoring JAVA_OPTS= -Duser.timezone=UTC
Please pass JVM parameters via ES_JAVA_OPTS instead
[2019-01-10T14:48:33,949][INFO ][o.e.n.Node ] [] initializing ...
[2019-01-10T14:48:34,022][INFO ][o.e.e.NodeEnvironment ] [VjV1ztb] using [1] data paths, mounts [[/ (/dev/vda1)]], net usable_space [34.7gb], net total_space [144.6gb], spins? [possibly], types [ext4]
[2019-01-10T14:48:34,023][INFO ][o.e.e.NodeEnvironment ] [VjV1ztb] heap size [1.9gb], compressed ordinary object pointers [true]
[2019-01-10T14:48:34,024][INFO ][o.e.n.Node ] node name [VjV1ztb] derived from node ID [VjV1ztb1TDm5L2vGGGektw]; set [node.name] to override
[2019-01-10T14:48:34,025][INFO ][o.e.n.Node ] version[5.6.4], pid[16488], build[8bbedf5/2017-10-31T18:55:38.105Z], OS[Linux/4.19.8-arch1-1-ARCH/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_192/25.192-b26]
[2019-01-10T14:48:34,025][INFO ][o.e.n.Node ] JVM arguments [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djdk.io.permissionsUseCanonicalPath=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j.skipJansi=true, -XX:+HeapDumpOnOutOfMemoryError, -Des.path.home=/opt/datafari/elk/elasticsearch]
[2019-01-10T14:48:34,818][INFO ][o.e.p.PluginsService ] [VjV1ztb] loaded module [aggs-matrix-stats]
[2019-01-10T14:48:34,818][INFO ][o.e.p.PluginsService ] [VjV1ztb] loaded module [ingest-common]
[2019-01-10T14:48:34,819][INFO ][o.e.p.PluginsService ] [VjV1ztb] loaded module [lang-expression]
[2019-01-10T14:48:34,819][INFO ][o.e.p.PluginsService ] [VjV1ztb] loaded module [lang-groovy]
[2019-01-10T14:48:34,819][INFO ][o.e.p.PluginsService ] [VjV1ztb] loaded module [lang-mustache]
[2019-01-10T14:48:34,819][INFO ][o.e.p.PluginsService ] [VjV1ztb] loaded module [lang-painless]
[2019-01-10T14:48:34,820][INFO ][o.e.p.PluginsService ] [VjV1ztb] loaded module [parent-join]
[2019-01-10T14:48:34,820][INFO ][o.e.p.PluginsService ] [VjV1ztb] loaded module [percolator]
[2019-01-10T14:48:34,820][INFO ][o.e.p.PluginsService ] [VjV1ztb] loaded module [reindex]
[2019-01-10T14:48:34,820][INFO ][o.e.p.PluginsService ] [VjV1ztb] loaded module [transport-netty3]
[2019-01-10T14:48:34,821][INFO ][o.e.p.PluginsService ] [VjV1ztb] loaded module [transport-netty4]
[2019-01-10T14:48:34,821][INFO ][o.e.p.PluginsService ] [VjV1ztb] no plugins loaded
[2019-01-10T14:48:36,054][INFO ][o.e.d.DiscoveryModule ] [VjV1ztb] using discovery type [zen]
[2019-01-10T14:48:36,483][INFO ][o.e.n.Node ] initialized
[2019-01-10T14:48:36,484][INFO ][o.e.n.Node ] [VjV1ztb] starting ...
[2019-01-10T14:48:36,602][INFO ][o.e.t.TransportService ] [VjV1ztb] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
[2019-01-10T14:48:36,612][WARN ][o.e.b.BootstrapChecks ] [VjV1ztb] max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
[2019-01-10T14:48:36,612][WARN ][o.e.b.BootstrapChecks ] [VjV1ztb] max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2019-01-10T14:48:39,660][INFO ][o.e.c.s.ClusterService ] [VjV1ztb] new_master {VjV1ztb}{VjV1ztb1TDm5L2vGGGektw}{cuzwRgn_Q4yo3c-5wj0txw}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)
[2019-01-10T14:48:39,684][INFO ][o.e.g.GatewayService ] [VjV1ztb] recovered [0] indices into cluster_state
[2019-01-10T14:48:39,687][INFO ][o.e.h.n.Netty4HttpServerTransport] [VjV1ztb] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
[2019-01-10T14:48:39,687][INFO ][o.e.n.Node ] [VjV1ztb] started
Elasticsearch startup completed successfully --- OK
Starting Kibana...
Kibana started !
Checking if Kibana is up and running ...
/opt/datafari/elk/scripts/elk-utils.sh: connect: Connection refused
/opt/datafari/elk/scripts/elk-utils.sh: line 92: /dev/tcp/localhost/5601: Connection refused
Kibana doesn't reply to requests on port 5601. Sleeping for a while and trying again... retry 1
[2019-01-10T14:48:54,627][INFO ][o.e.c.m.MetaDataCreateIndexService] [VjV1ztb] [.kibana] creating index, cause [api], templates [], shards [1]/[1], mappings [default, index-pattern, server, visualization, search, timelion-sheet, config, dashboard, url]
/opt/datafari/elk/scripts/elk-utils.sh: connect: Connection refused
/opt/datafari/elk/scripts/elk-utils.sh: line 104: /dev/tcp/localhost/5601: Connection refused
Kibana doesn't reply to requests on port 5601. Sleeping for a while and trying again... retry 2
/opt/datafari/elk/scripts/elk-utils.sh: connect: Connection refused
/opt/datafari/elk/scripts/elk-utils.sh: line 104: /dev/tcp/localhost/5601: Connection refused
Kibana doesn't reply to requests on port 5601. Sleeping for a while and trying again... retry 3
/opt/datafari/elk/scripts/elk-utils.sh: connect: Connection refused
/opt/datafari/elk/scripts/elk-utils.sh: line 104: /dev/tcp/localhost/5601: Connection refused
Kibana doesn't reply to requests on port 5601. Sleeping for a while and trying again... retry 4
/opt/datafari/elk/scripts/elk-utils.sh: connect: Connection refused
/opt/datafari/elk/scripts/elk-utils.sh: line 104: /dev/tcp/localhost/5601: Connection refused
Kibana doesn't reply to requests on port 5601. Sleeping for a while and trying again... retry 5
/opt/datafari/elk/scripts/elk-utils.sh: connect: Connection refused
/opt/datafari/elk/scripts/elk-utils.sh: line 104: /dev/tcp/localhost/5601: Connection refused
/!\ ERROR: Kibana startup has ended with errors

kibana.log:

{"type":"log","@timestamp":"2019-01-10T13:48:49Z","tags":["status","plugin:kibana@5.6.4","info"],"pid":16579,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
{"type":"log","@timestamp":"2019-01-10T13:48:49Z","tags":["status","plugin:elasticsearch@5.6.4","info"],"pid":16579,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
{"type":"log","@timestamp":"2019-01-10T13:48:49Z","tags":["status","plugin:console@5.6.4","info"],"pid":16579,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
{"type":"log","@timestamp":"2019-01-10T13:48:49Z","tags":["status","plugin:metrics@5.6.4","info"],"pid":16579,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
{"type":"log","@timestamp":"2019-01-10T13:48:49Z","tags":["status","plugin:timelion@5.6.4","info"],"pid":16579,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
{"type":"log","@timestamp":"2019-01-10T13:48:49Z","tags":["listening","info"],"pid":16579,"message":"Server running at http://XXXXXXXXXX:5601"}
{"type":"log","@timestamp":"2019-01-10T13:48:49Z","tags":["status","ui settings","info"],"pid":16579,"state":"yellow","message":"Status changed from uninitialized to yellow - Elasticsearch plugin is yellow","prevState":"uninitialized","prevMsg":"uninitialized"}
{"type":"log","@timestamp":"2019-01-10T13:48:54Z","tags":["status","plugin:elasticsearch@5.6.4","info"],"pid":16579,"state":"yellow","message":"Status changed from yellow to yellow - No existing Kibana index found","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
{"type":"log","@timestamp":"2019-01-10T13:48:55Z","tags":["status","plugin:elasticsearch@5.6.4","info"],"pid":16579,"state":"green","message":"Status changed from yellow to green - Kibana index ready","prevState":"yellow","prevMsg":"No existing Kibana index found"}
{"type":"log","@timestamp":"2019-01-10T13:48:55Z","tags":["status","ui settings","info"],"pid":16579,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Elasticsearch plugin is yellow"}

Hence, the index initialization fails.

Answer 1 · 2019-01-10T21:36:57.000Z

Hi Marmelion,

we will soon close the possibility to create issues in github, in order to focus on jira. Can you create a jira account and create the corresponding ticket ? Here is the login page: https://datafari.atlassian.net/login.jsp?os_destination=https%3A%2F%2Fdatafari.atlassian.net%2Fprojects%2FDATAFARI%2Fissues%2FDATAFARI-319%3Ffilter%3Dallopenissues

If you don't have an account already, the creation is free.