documentation bug at transform->mapping: https://pgsync.com/tutorial/transform-nodes/
MeRuslan opened this issue · 4 comments
MeRuslan commented
PGSync version:
3.1.0
Postgres version:
PostgreSQL 13.12
Elasticsearch/OpenSearch version:
elasticsearch:8.10.2
Redis version:
NA
Python version:
3.9
Problem Description:
Minimal example schema:
[
{
"database": "postgres",
"index": "user_index",
"nodes": {
"table": "user",
"columns": [
"id"
],
"transform": {
"mapping": {
"id": "long"
}
}
}
}
]
Works well without transform->mapping
Error Message (if any):
2024-05-06 18:51:06 .env found!
2024-05-06 18:51:06 localhost:9432
2024-05-06 18:51:06 localhost:9201
2024-05-06 18:51:06 localhost:6389
2024-05-06 18:51:08 - public.user
2024-05-06 18:51:09 - public.user
2024-05-06 18:51:09 0:00:00.252404 (0.25 sec)
2024-05-06 18:51:06 wait-for-it.sh: waiting 30 seconds for localhost:9432
2024-05-06 18:51:06 wait-for-it.sh: localhost:9432 is available after 0 seconds
2024-05-06 18:51:06 wait-for-it.sh: waiting 30 seconds for localhost:9201
2024-05-06 18:51:06 wait-for-it.sh: localhost:9201 is available after 0 seconds
2024-05-06 18:51:06 wait-for-it.sh: waiting 30 seconds for localhost:6389
2024-05-06 18:51:06 wait-for-it.sh: localhost:6389 is available after 0 seconds
2024-05-06 18:51:08 Traceback (most recent call last):
2024-05-06 18:51:08 File "/app/.venv/bin/bootstrap", line 75, in <module>
2024-05-06 18:51:08 main()
2024-05-06 18:51:08 File "/app/.venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
2024-05-06 18:51:08 return self.main(*args, **kwargs)
2024-05-06 18:51:08 ^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-06 18:51:08 File "/app/.venv/lib/python3.11/site-packages/click/core.py", line 1078, in main
2024-05-06 18:51:08 rv = self.invoke(ctx)
2024-05-06 18:51:08 ^^^^^^^^^^^^^^^^
2024-05-06 18:51:08 File "/app/.venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
2024-05-06 18:51:08 return ctx.invoke(self.callback, **ctx.params)
2024-05-06 18:51:08 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-06 18:51:08 File "/app/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
2024-05-06 18:51:08 return __callback(*args, **kwargs)
2024-05-06 18:51:08 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-06 18:51:08 File "/app/.venv/bin/bootstrap", line 60, in main
2024-05-06 18:51:08 sync: Sync = Sync(
2024-05-06 18:51:08 ^^^^^
2024-05-06 18:51:08 File "/app/.venv/lib/python3.11/site-packages/pgsync/singleton.py", line 36, in __call__
2024-05-06 18:51:08 cls._instances[key] = super(Singleton, cls).__call__(
2024-05-06 18:51:08 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-06 18:51:08 File "/app/.venv/lib/python3.11/site-packages/pgsync/sync.py", line 105, in __init__
2024-05-06 18:51:08 self.create_setting()
2024-05-06 18:51:08 File "/app/.venv/lib/python3.11/site-packages/pgsync/sync.py", line 252, in create_setting
2024-05-06 18:51:08 self.search_client._create_setting(
2024-05-06 18:51:08 File "/app/.venv/lib/python3.11/site-packages/pgsync/search_client.py", line 277, in _create_setting
2024-05-06 18:51:08 mapping = self._build_mapping(tree, routing)
2024-05-06 18:51:08 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-06 18:51:08 File "/app/.venv/lib/python3.11/site-packages/pgsync/search_client.py", line 309, in _build_mapping
2024-05-06 18:51:08 column_type: str = mapping[column]["type"]
2024-05-06 18:51:08 ~~~~~~~~~~~~~~~^^^^^^^^
2024-05-06 18:51:08 TypeError: string indices must be integers, not 'str'
2024-05-06 18:51:09 Traceback (most recent call last):
2024-05-06 18:51:09 File "/app/.venv/bin/pgsync", line 7, in <module>
2024-05-06 18:51:09 sync.main()
2024-05-06 18:51:09 File "/app/.venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
2024-05-06 18:51:09 return self.main(*args, **kwargs)
2024-05-06 18:51:09 ^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-06 18:51:09 File "/app/.venv/lib/python3.11/site-packages/click/core.py", line 1078, in main
2024-05-06 18:51:09 rv = self.invoke(ctx)
2024-05-06 18:51:09 ^^^^^^^^^^^^^^^^
2024-05-06 18:51:09 File "/app/.venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
2024-05-06 18:51:09 return ctx.invoke(self.callback, **ctx.params)
2024-05-06 18:51:09 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-06 18:51:09 File "/app/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
2024-05-06 18:51:09 return __callback(*args, **kwargs)
2024-05-06 18:51:09 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-06 18:51:09 File "/app/.venv/lib/python3.11/site-packages/pgsync/sync.py", line 1480, in main
2024-05-06 18:51:09 sync: Sync = Sync(
2024-05-06 18:51:09 ^^^^^
2024-05-06 18:51:09 File "/app/.venv/lib/python3.11/site-packages/pgsync/singleton.py", line 36, in __call__
2024-05-06 18:51:09 cls._instances[key] = super(Singleton, cls).__call__(
2024-05-06 18:51:09 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-06 18:51:09 File "/app/.venv/lib/python3.11/site-packages/pgsync/sync.py", line 105, in __init__
2024-05-06 18:51:09 self.create_setting()
2024-05-06 18:51:09 File "/app/.venv/lib/python3.11/site-packages/pgsync/sync.py", line 252, in create_setting
2024-05-06 18:51:09 self.search_client._create_setting(
2024-05-06 18:51:09 File "/app/.venv/lib/python3.11/site-packages/pgsync/search_client.py", line 277, in _create_setting
2024-05-06 18:51:09 mapping = self._build_mapping(tree, routing)
2024-05-06 18:51:09 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-06 18:51:09 File "/app/.venv/lib/python3.11/site-packages/pgsync/search_client.py", line 309, in _build_mapping
2024-05-06 18:51:09 column_type: str = mapping[column]["type"]
2024-05-06 18:51:09 ~~~~~~~~~~~~~~~^^^^^^^^
2024-05-06 18:51:09 TypeError: string indices must be integers, not 'str'
MeRuslan commented
am I doing something wrong?
SkymaxPlay commented
Correct format:
"transform": {
"mapping": {
"id": {
"type": "long"
}
}
}MeRuslan commented
@SkymaxPlay thanks!
There's a documentation issue then at https://pgsync.com/tutorial/transform-nodes/
toluaina commented
This has been fixed, thanks and apologies for the delay resolving this.