singer-io/tap-postgres

hstore_to_array() does not escape single quotes in value

Closed this issue · 0 comments

drdee commented

We have a column with an hstore datatype in a table that we are replicating using logical replication.

The select command that is being issued is:

SELECT hstore_to_array('"nickname"=>"Dave's Courtyard"')

the single quote should be escaped because this throws the following error:

2019-07-03 13:57:31,042Z    tap - Traceback (most recent call last):
2019-07-03 13:57:31,042Z    tap -   File "tap-env/bin/tap-postgres", line 10, in <module>
2019-07-03 13:57:31,043Z    tap -     sys.exit(main())
2019-07-03 13:57:31,043Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_postgres/__init__.py", line 705, in main
2019-07-03 13:57:31,043Z    tap -     raise exc
2019-07-03 13:57:31,043Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_postgres/__init__.py", line 702, in main
2019-07-03 13:57:31,043Z    tap -     main_impl()
2019-07-03 13:57:31,043Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_postgres/__init__.py", line 696, in main_impl
2019-07-03 13:57:31,043Z    tap -     do_sync(conn_config, args.catalog.to_dict() if args.catalog else args.properties, args.config.get('default_replication_method'), state)
2019-07-03 13:57:31,043Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_postgres/__init__.py", line 671, in do_sync
2019-07-03 13:57:31,043Z    tap -     state = sync_logical_streams(conn_config, list(streams), state, end_lsn)
2019-07-03 13:57:31,043Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_postgres/__init__.py", line 581, in sync_logical_streams
2019-07-03 13:57:31,043Z    tap -     state = logical_replication.sync_tables(conn_config, logical_streams, state, end_lsn)
2019-07-03 13:57:31,043Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_postgres/sync_strategies/logical_replication.py", line 355, in sync_tables
2019-07-03 13:57:31,044Z    tap -     state = consume_message(logical_streams, state, msg, time_extracted, conn_info, end_lsn)
2019-07-03 13:57:31,044Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_postgres/sync_strategies/logical_replication.py", line 263, in consume_message
2019-07-03 13:57:31,044Z    tap -     record_message = row_to_singer_message(target_stream, col_vals, stream_version, col_names, time_extracted, stream_md_map, conn_info)
2019-07-03 13:57:31,044Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_postgres/sync_strategies/logical_replication.py", line 203, in row_to_singer_message
2019-07-03 13:57:31,044Z    tap -     cleaned_elem = selected_value_to_singer_value(elem, sql_datatype, conn_info)
2019-07-03 13:57:31,044Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_postgres/sync_strategies/logical_replication.py", line 189, in selected_value_to_singer_value
2019-07-03 13:57:31,044Z    tap -     return selected_value_to_singer_value_impl(elem, sql_datatype, conn_info)
2019-07-03 13:57:31,044Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_postgres/sync_strategies/logical_replication.py", line 165, in selected_value_to_singer_value_impl
2019-07-03 13:57:31,044Z    tap -     return create_hstore_elem(conn_info, elem)
2019-07-03 13:57:31,044Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_postgres/sync_strategies/logical_replication.py", line 75, in create_hstore_elem
2019-07-03 13:57:31,044Z    tap -     cur.execute(sql)
2019-07-03 13:57:31,044Z    tap - psycopg2.ProgrammingError: syntax error at or near "s"
2019-07-03 13:57:31,044Z    tap - LINE 1: SELECT hstore_to_array('"nickname"=>"Dave's Courtyard"')
2019-07-03 13:57:31,044Z    tap -                                                   ^