Issues when following tutorial about »InfluxDB data import«
matkuliak opened this issue · 2 comments
matkuliak commented
There seems to be issue when following the Import data from InfluxDB tutorial.
When using the Docker setup, ctk is returning Failed to establish a new connection: [Errno 111] Connection refused
Some logs below could help:
13:10:08 ~ $ docker run -d --name crate -p 4200:4200 crate/crate:latest
c1b49b007828ee1dd2254afae442a9ae8d1b73a1eea7e1de90f9df93b84c9cef
13:10:10 ~ $ docker run -d --name influxdb -p 8086:8086 \
--env=DOCKER_INFLUXDB_INIT_MODE=setup \
--env=DOCKER_INFLUXDB_INIT_USERNAME=user1 \
--env=DOCKER_INFLUXDB_INIT_PASSWORD=secret1234 \
--env=DOCKER_INFLUXDB_INIT_ORG=example \
--env=DOCKER_INFLUXDB_INIT_BUCKET=testdrive \
--env=DOCKER_INFLUXDB_INIT_ADMIN_TOKEN=token \
influxdb:latest
d91a06501341315907c9b2cdd4d005473361ef38f05c705802b85457ee52d017
13:10:21 ~ $ docker exec influxdb influx write --bucket=testdrive --org=example --precision=s --token=token "demo,region=amazonas temperature=27.4,humidity=92.3,windspeed=4.5 1588363200"
docker exec influxdb influx write --bucket=testdrive --org=example --precision=s --token=token "demo,region=amazonas temperature=28.2,humidity=88.7,windspeed=4.7 1588549600"
docker exec influxdb influx write --bucket=testdrive --org=example --precision=s --token=token "demo,region=amazonas temperature=27.9,humidity=91.6,windspeed=3.2 1588736000"
docker exec influxdb influx write --bucket=testdrive --org=example --precision=s --token=token "demo,region=amazonas temperature=29.1,humidity=88.1,windspeed=2.4 1588922400"
docker exec influxdb influx write --bucket=testdrive --org=example --precision=s --token=token "demo,region=amazonas temperature=28.6,humidity=93.4,windspeed=2.9 1589108800"
13:11:00 ~ $ alias crash="docker run --rm -it ghcr.io/crate-workbench/cratedb-toolkit:latest crash"
alias ctk="docker run --rm -it ghcr.io/crate-workbench/cratedb-toolkit:latest ctk"
13:11:16 ~ $ export CRATEDB_SQLALCHEMY_URL=crate://crate@localhost:4200/testdrive/demo
13:11:49 ~ $ ctk load table influxdb2://example:token@localhost:8086/testdrive/demo
Traceback (most recent call last):
File "/usr/local/bin/ctk", line 8, in <module>
sys.exit(cli())
^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/cratedb_toolkit/io/cli.py", line 61, in load_table
raise KeyError(error_message)
KeyError: 'Either CrateDB Cloud Cluster identifier or CrateDB SQLAlchemy or HTTP URL needs to be supplied. Use --cluster-id / --cratedb-sqlalchemy-url / --cratedb-http-url CLI options or CRATEDB_CLOUD_CLUSTER_ID / CRATEDB_SQLALCHEMY_URL / CRATEDB_HTTP_URL environment variables.'
//making alias didn't work, so I supplement it with "--cratedb-sqlalchemy-url"
13:12:55 ~ $ ctk load table influxdb2://example:token@localhost:8086/testdrive/demo --cratedb-sqlalchemy-url "crate://crate@localhost:4200/testdrive/demo"
2024-05-28 11:12:57,570 [cratedb_toolkit.io.influxdb ] INFO : Running InfluxDB copy
2024-05-28 11:12:57,573 [influxio.core ] INFO : Copying from http://example:token@localhost:8086/testdrive/demo to crate://crate@localhost:4200/testdrive/demo
2024-05-28 11:12:57,573 [influxio.adapter ] INFO : SQLAlchemy DB URI: crate://crate@localhost:4200/?schema=testdrive
2024-05-28 11:12:57,583 [influxio.adapter ] INFO : Loading dataframes into RDBMS/SQL database using pandas/Dask
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 203, in _new_conn
sock = connection.create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
raise err
File "/usr/local/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/bin/ctk", line 8, in <module>
sys.exit(cli())
^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/cratedb_toolkit/io/cli.py", line 85, in load_table
return cluster.load_table(resource=resource, target=target)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/cratedb_toolkit/api/main.py", line 113, in load_table
if not influxdb_copy(source_url, target_url, progress=True):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/cratedb_toolkit/io/influxdb.py", line 16, in influxdb_copy
copy(source_url, target_url, progress=progress)
File "/usr/local/lib/python3.11/site-packages/influxio/core.py", line 90, in copy
sink.write(source_node)
File "/usr/local/lib/python3.11/site-packages/influxio/adapter.py", line 298, in write
for df in source.read_df():
File "/usr/local/lib/python3.11/site-packages/influxio/adapter.py", line 68, in read_df
for df in self.client.query_api().query_data_frame_stream(query=query):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/influxdb_client/client/query_api.py", line 299, in query_data_frame_stream
response = self._query_api.post_query(org=org, query=self._create_query(query, self.default_dialect, params,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/influxdb_client/service/query_service.py", line 285, in post_query
(data) = self.post_query_with_http_info(**kwargs) # noqa: E501
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/influxdb_client/service/query_service.py", line 311, in post_query_with_http_info
return self.api_client.call_api(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/influxdb_client/_sync/api_client.py", line 343, in call_api
return self.__call_api(resource_path, method,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/influxdb_client/_sync/api_client.py", line 173, in __call_api
response_data = self.request(
^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/influxdb_client/_sync/api_client.py", line 388, in request
return self.rest_client.POST(url,
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/influxdb_client/_sync/rest.py", line 311, in POST
return self.request("POST", url,
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/influxdb_client/_sync/rest.py", line 186, in request
r = self.pool_manager.request(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/urllib3/_request_methods.py", line 118, in request
return self.request_encode_body(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/urllib3/_request_methods.py", line 217, in request_encode_body
return self.urlopen(method, url, **extra_kw)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/urllib3/poolmanager.py", line 444, in urlopen
response = conn.urlopen(method, u.request_uri, **kw)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 844, in urlopen
retries = retries.increment(
^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/urllib3/util/retry.py", line 445, in increment
raise reraise(type(error), error, _stacktrace)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/urllib3/util/util.py", line 39, in reraise
raise value
File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 790, in urlopen
response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 496, in _make_request
conn.request(
File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 395, in request
self.endheaders()
File "/usr/local/lib/python3.11/http/client.py", line 1298, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.11/http/client.py", line 1058, in _send_output
self.send(msg)
File "/usr/local/lib/python3.11/http/client.py", line 996, in send
self.connect()
File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 243, in connect
self.sock = self._new_conn()
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 218, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f9ebfd32450>: Failed to establish a new connection: [Errno 111] Connection refused
amotl commented
Thanks for reporting. I assumed the steps in that patch would have been verified. Apologies for not double checking them.
Please adjust the tutorial according to the suggestions added to those issue tickets. Thanks!