Bug Report - Failed to establish new connection: [Errno 111] connection refused
davidpr opened this issue · 1 comments
After having run the docker-compose -f docker-amundsen-atlas.yml up
I get this:
amundsensearch | 2022-12-08T21:18:03+0000.551 [INFO] _internal._log:225 (1:MainThread) - * Running on http://172.20.0.5:5001/ (Press CTRL+C to quit)
es_amundsen_atlas | [2022-12-08T21:18:04,041][INFO ][o.e.e.NodeEnvironment ] [vsUmzuE] using [1] data paths, mounts [[/ (overlay)]], net usable_space [29.3gb], net total_space [97.8gb], types [overlay]
es_amundsen_atlas | [2022-12-08T21:18:04,056][INFO ][o.e.e.NodeEnvironment ] [vsUmzuE] heap size [1gb], compressed ordinary object pointers [true]
es_amundsen_atlas | [2022-12-08T21:18:04,068][INFO ][o.e.n.Node ] [vsUmzuE] node name derived from node ID [vsUmzuE3SqeGEiEm3lNNRQ]; set [node.name] to override
es_amundsen_atlas | [2022-12-08T21:18:04,069][INFO ][o.e.n.Node ] [vsUmzuE] version[6.8.13], pid[1], build[default/docker/be13c69/2020-10-16T09:09:46.555371Z], OS[Linux/5.15.0-56-generic/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/15/15+36]
es_amundsen_atlas | [2022-12-08T21:18:04,072][INFO ][o.e.n.Node ] [vsUmzuE] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-1350448486499493754, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -XX:UseAVX=2, -Des.cgroups.hierarchy.override=/, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=docker]
es_amundsen_atlas | [2022-12-08T21:18:08,576][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [aggs-matrix-stats]
es_amundsen_atlas | [2022-12-08T21:18:08,576][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [analysis-common]
es_amundsen_atlas | [2022-12-08T21:18:08,576][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [ingest-common]
es_amundsen_atlas | [2022-12-08T21:18:08,576][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [ingest-geoip]
es_amundsen_atlas | [2022-12-08T21:18:08,576][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [ingest-user-agent]
es_amundsen_atlas | [2022-12-08T21:18:08,577][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [lang-expression]
es_amundsen_atlas | [2022-12-08T21:18:08,577][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [lang-mustache]
es_amundsen_atlas | [2022-12-08T21:18:08,577][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [lang-painless]
es_amundsen_atlas | [2022-12-08T21:18:08,577][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [mapper-extras]
es_amundsen_atlas | [2022-12-08T21:18:08,577][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [parent-join]
es_amundsen_atlas | [2022-12-08T21:18:08,577][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [percolator]
es_amundsen_atlas | [2022-12-08T21:18:08,577][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [rank-eval]
es_amundsen_atlas | [2022-12-08T21:18:08,577][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [reindex]
es_amundsen_atlas | [2022-12-08T21:18:08,578][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [repository-url]
es_amundsen_atlas | [2022-12-08T21:18:08,578][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [transport-netty4]
es_amundsen_atlas | [2022-12-08T21:18:08,580][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [tribe]
es_amundsen_atlas | [2022-12-08T21:18:08,580][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [x-pack-ccr]
es_amundsen_atlas | [2022-12-08T21:18:08,580][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [x-pack-core]
es_amundsen_atlas | [2022-12-08T21:18:08,581][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [x-pack-deprecation]
es_amundsen_atlas | [2022-12-08T21:18:08,581][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [x-pack-graph]
es_amundsen_atlas | [2022-12-08T21:18:08,581][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [x-pack-ilm]
es_amundsen_atlas | [2022-12-08T21:18:08,585][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [x-pack-logstash]
es_amundsen_atlas | [2022-12-08T21:18:08,586][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [x-pack-ml]
es_amundsen_atlas | [2022-12-08T21:18:08,586][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [x-pack-monitoring]
es_amundsen_atlas | [2022-12-08T21:18:08,586][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [x-pack-rollup]
es_amundsen_atlas | [2022-12-08T21:18:08,586][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [x-pack-security]
es_amundsen_atlas | [2022-12-08T21:18:08,586][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [x-pack-sql]
es_amundsen_atlas | [2022-12-08T21:18:08,586][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [x-pack-upgrade]
es_amundsen_atlas | [2022-12-08T21:18:08,587][INFO ][o.e.p.PluginsService ] [vsUmzuE] loaded module [x-pack-watcher]
es_amundsen_atlas | [2022-12-08T21:18:08,587][INFO ][o.e.p.PluginsService ] [vsUmzuE] no plugins loaded
es_amundsen_atlas | [2022-12-08T21:18:19,223][INFO ][o.e.x.s.a.s.FileRolesStore] [vsUmzuE] parsed [0] roles from file [/usr/share/elasticsearch/config/roles.yml]
es_amundsen_atlas | [2022-12-08T21:18:20,943][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [vsUmzuE] [controller/96] [Main.cc@114] controller (64 bit): Version 6.8.13 (Build c959f131d0de3b) Copyright (c) 2020 Elasticsearch BV
es_amundsen_atlas | [2022-12-08T21:18:22,462][INFO ][o.e.d.DiscoveryModule ] [vsUmzuE] using discovery type [zen] and host providers [settings]
es_amundsen_atlas | [2022-12-08T21:18:25,119][INFO ][o.e.n.Node ] [vsUmzuE] initialized
es_amundsen_atlas | [2022-12-08T21:18:25,120][INFO ][o.e.n.Node ] [vsUmzuE] starting ...
es_amundsen_atlas | [2022-12-08T21:18:25,405][INFO ][o.e.t.TransportService ] [vsUmzuE] publish_address {172.20.0.3:9300}, bound_addresses {0.0.0.0:9300}
es_amundsen_atlas | [2022-12-08T21:18:25,448][INFO ][o.e.b.BootstrapChecks ] [vsUmzuE] bound or publishing to a non-loopback address, enforcing bootstrap checks
es_amundsen_atlas | ERROR: [1] bootstrap checks failed
es_amundsen_atlas | [1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
es_amundsen_atlas | [2022-12-08T21:18:25,477][INFO ][o.e.n.Node ] [vsUmzuE] stopping ...
es_amundsen_atlas | [2022-12-08T21:18:25,574][INFO ][o.e.n.Node ] [vsUmzuE] stopped
es_amundsen_atlas | [2022-12-08T21:18:25,574][INFO ][o.e.n.Node ] [vsUmzuE] closing ...
es_amundsen_atlas | [2022-12-08T21:18:25,627][INFO ][o.e.n.Node ] [vsUmzuE] closed
es_amundsen_atlas | [2022-12-08T21:18:25,636][INFO ][o.e.x.m.p.NativeController] [vsUmzuE] Native controller process has stopped - no new native processes can be started
es_amundsen_atlas exited with code 78
atlas | The Server is no longer running with pid 801
atlas | configured for local hbase.
atlas | hbase started.
atlas | configured for local solr.
atlas | solr started.
atlas | setting up solr collections...
atlas | starting atlas on host localhost
atlas | starting atlas on port 21000
atlas | .............................................
atlas | Apache Atlas Server started!!!
atlas |
atlas | Creating ingestion-source type...
atlas | % Total % Received % Xferd Average Speed Time Time Time Current
atlas | Dload Upload Total Spent Left Speed
100 1263 0 88 100 1175 118 1585 --:--:-- --:--:-- --:--:-- 1585
atlas | HTTP/1.1 100 Continue
atlas |
atlas | HTTP/1.1 409 Conflict
atlas | Date: Thu, 08 Dec 2022 21:19:17 GMT
atlas | Set-Cookie: ATLASSESSIONID=19jllwd7hlt8s1lmc8yi429oy5;Path=/;HttpOnly
atlas | Expires: Thu, 01 Jan 1970 00:00:00 GMT
atlas | X-Frame-Options: DENY
atlas | X-Content-Type-Options: nosniff
atlas | X-XSS-Protection: 1; mode=block
atlas | Strict-Transport-Security: max-age=31536000; includeSubDomains
atlas | Content-Type: application/json; charset=UTF-8
atlas | X-Content-Type-Options: nosniff
atlas | Content-Security-Policy: default-src 'self'; script-src 'self' 'unsafe-inline' 'unsafe-eval' blob: data:; connect-src 'self'; img-src 'self' blob: data:; style-src 'self' 'unsafe-inline';font-src 'self' data:
atlas | Server: Apache Atlas
atlas | Transfer-Encoding: chunked
atlas |
atlas | {"errorCode":"ATLAS-409-00-001","errorMessage":"Given type staging_node already exists"}
atlas | ingestion-source created
atlas | Creating file type...
atlas | % Total % Received % Xferd Average Speed Time Time Time Current
atlas | Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0HTTP/1.1 409 Conflict
atlas | Date: Thu, 08 Dec 2022 21:19:17 GMT
then at the moment of doing the
python3 example/scripts/sample_data_loader_atlas.py
I get these errors:
2022-12-09 06:07:21,608 elasticsearch WARNING GET http://localhost:9200/ [status:N/A request:0.001s]
Traceback (most recent call last):
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/urllib3/connection.py", line 174, in _new_conn
conn = connection.create_connection(
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/urllib3/util/connection.py", line 95, in create_connection
raise err
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/urllib3/util/connection.py", line 85, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/elasticsearch/connection/http_urllib3.py", line 255, in perform_request
response = self.pool.urlopen(
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen
retries = retries.increment(
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/urllib3/util/retry.py", line 525, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/urllib3/packages/six.py", line 770, in reraise
raise value
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 398, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/urllib3/connection.py", line 239, in request
super(HTTPConnection, self).request(method, url, body=body, headers=headers)
File "/usr/lib/python3.10/http/client.py", line 1282, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.10/http/client.py", line 1328, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.10/http/client.py", line 1277, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.10/http/client.py", line 1037, in _send_output
self.send(msg)
File "/usr/lib/python3.10/http/client.py", line 975, in send
self.connect()
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/urllib3/connection.py", line 205, in connect
conn = self._new_conn()
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/urllib3/connection.py", line 186, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fa47c9198a0>: Failed to establish a new connection: [Errno 111] Connection refused
2022-12-09 06:07:21,617 databuilder.callback.call_back INFO No callbacks to notify
Traceback (most recent call last):
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/urllib3/connection.py", line 174, in _new_conn
conn = connection.create_connection(
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/urllib3/util/connection.py", line 95, in create_connection
raise err
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/urllib3/util/connection.py", line 85, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/elasticsearch/connection/http_urllib3.py", line 255, in perform_request
response = self.pool.urlopen(
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen
retries = retries.increment(
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/urllib3/util/retry.py", line 525, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/urllib3/packages/six.py", line 770, in reraise
raise value
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 398, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/urllib3/connection.py", line 239, in request
super(HTTPConnection, self).request(method, url, body=body, headers=headers)
File "/usr/lib/python3.10/http/client.py", line 1282, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.10/http/client.py", line 1328, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.10/http/client.py", line 1277, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.10/http/client.py", line 1037, in _send_output
self.send(msg)
File "/usr/lib/python3.10/http/client.py", line 975, in send
self.connect()
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/urllib3/connection.py", line 205, in connect
conn = self._new_conn()
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/urllib3/connection.py", line 186, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fa47c9198a0>: Failed to establish a new connection: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/david/amundsen-tests/amundsen/databuilder/example/scripts/sample_data_loader_atlas.py", line 406, in
job_es_table.launch()
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/amundsen_databuilder-7.4.2-py3.10.egg/databuilder/job/job.py", line 76, in launch
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/amundsen_databuilder-7.4.2-py3.10.egg/databuilder/job/job.py", line 72, in launch
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/amundsen_databuilder-7.4.2-py3.10.egg/databuilder/publisher/base_publisher.py", line 40, in publish
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/amundsen_databuilder-7.4.2-py3.10.egg/databuilder/publisher/base_publisher.py", line 37, in publish
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/amundsen_databuilder-7.4.2-py3.10.egg/databuilder/publisher/elasticsearch_publisher.py", line 104, in publish_impl
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/elasticsearch/client/utils.py", line 347, in _wrapped
return func(*args, params=params, headers=headers, **kwargs)
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/elasticsearch/client/indices.py", line 145, in create
return self.transport.perform_request(
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/elasticsearch/transport.py", line 417, in perform_request
self._do_verify_elasticsearch(headers=headers, timeout=timeout)
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/elasticsearch/transport.py", line 606, in _do_verify_elasticsearch
raise error
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/elasticsearch/transport.py", line 569, in _do_verify_elasticsearch
_, info_headers, info_response = conn.perform_request(
File "/home/david/amundsen-tests/amundsen/databuilder/venv/lib/python3.10/site-packages/elasticsearch/connection/http_urllib3.py", line 280, in perform_request
raise ConnectionError("N/A", str(e), e)
elasticsearch.exceptions.ConnectionError: ConnectionError(<urllib3.connection.HTTPConnection object at 0x7fa47c9198a0>: Failed to establish a new connection: [Errno 111] Connection refused) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object at 0x7fa47c9198a0>: Failed to establish a new connection: [Errno 111] Connection refused)
Could you make me some sense of what is happening and how can it be solved?
thanks,
David
Thanks for opening your first issue here!