plpython3 extension installation
Closed this issue · 4 comments
bachpro900 commented
Hello,
In advance, thank you for your help.
I am not able to install plpython extension.
Installation version is : BABELFISH_VERSION=BABEL_3_2_0__PG_15_3
I added these lines the Dockerfile in order to have python3 headers:
apt update && apt install -y\
software-properties-common &&\
add-apt-repository ppa:deadsnakes/ppa
RUN apt install -y python3-dev libpq-dev\
And these lines to compile the extension:
WORKDIR ${PG_SRC}/src/pl
RUN make && make install
WORKDIR ${PG_SRC}/src/pl/plpython
RUN make && make PG_CONFIG=${PG_CONFIG} install
But still get below issue (seems that the headers are not present, or explicite path must be specified, or the version is not OK ...). In all bablefish documentation and installation, nothing found about python version to get the extension installed.
Here the error:
1.016 /usr/bin/msgfmt -c -o po/zh_CN.mo po/zh_CN.po
1.031 gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -g -ggdb -fPIC -I. -I. -I../../../src/include -D_GNU_SOURCE -I/usr/include/libxml2 -c -o plpy_cursorobject.o plpy_cursorobject.c
1.235 In file included from plpy_typeio.h:10,
1.235 from plpy_cursorobject.h:8,
1.235 from plpy_cursorobject.c:14:
1.235 plpython.h:59:10: fatal error: Python.h: No such file or directory
1.235 59 | #include <Python.h>
1.235 | ^~~~~~~~~~
1.238 compilation terminated.
1.243 make: *** [<builtin>: plpy_cursorobject.o] Error 1
------
Dockerfile:118
--------------------
116 | RUN make && make install
117 | WORKDIR ${PG_SRC}/src/pl/plpython
118 | >>> RUN make && make PG_CONFIG=${PG_CONFIG} install
119 |
120 |
--------------------
ERROR: failed to solve: process "/bin/sh -c make && make PG_CONFIG=${PG_CONFIG} install" did not complete successfully: exit code: 2
jonathanpotts commented
The following commit adds the plpython3u extension to the build:
acfeda3
bachpro900 commented
Hello,
Plpythonu is present in pg_available_extensions table, plpythonu.control and plpythonu--1.0.sql files are present in '/opt/babelfish/share/postgresql/extension/' directory, but the installation of the extension does not work. Still plpython3u.control is missing in '/opt/babelfish/lib/postgresql/
' directory.
babelfish_db=# create extension plpython3u;
ERROR: could not open extension control file "/opt/babelfish/share/postgresql/extension/plpython3u.control": No such file or directory
babelfish_db=# create extension plpython2u;
ERROR: could not load library "/opt/babelfish/lib/postgresql/plpython2.so": libpython2.7.so.1.0: cannot open shared object file: No such file or directory
babelfish_db=# create extension plpythonu;
ERROR: could not load library "/opt/babelfish/lib/postgresql/plpython2.so": libpython2.7.so.1.0: cannot open shared object file: No such file or directory
babelfish_db=#
jonathanpotts commented
It works on my end.
Please try running the following commands (which use the commit I previously mentioned):
docker build -t jonathanpotts/babelfishpg:plpython3u https://github.com/jonathanpotts/docker-babelfishpg.git#acfeda30b147d4f53659c429df08c3b5882d4b8a --build-arg BABELFISH_VERSION=BABEL_3_2_0__PG_15_3
docker run -d -p 1433:1433 jonathanpotts/babelfishpg:plpython3uWhen I do:
I get the following result (which creates the plpython3u extension and shows no errors for plpython3u):
2023-11-02 11:57:48 The files belonging to this database system will be owned by user "postgres".
2023-11-02 11:57:48 This user must also own the server process.
2023-11-02 11:57:48
2023-11-02 11:57:48 The database cluster will be initialized with locale "C".
2023-11-02 11:57:48 The default text search configuration will be set to "english".
2023-11-02 11:57:48
2023-11-02 11:57:48 Data page checksums are disabled.
2023-11-02 11:57:48
2023-11-02 11:57:48 fixing permissions on existing directory /data/babelfish ... ok
2023-11-02 11:57:48 creating subdirectories ... ok
2023-11-02 11:57:48 selecting dynamic shared memory implementation ... posix
2023-11-02 11:57:48 selecting default max_connections ... 100
2023-11-02 11:57:48 selecting default shared_buffers ... 128MB
2023-11-02 11:57:48 selecting default time zone ... Etc/UTC
2023-11-02 11:57:48 creating configuration files ... ok
2023-11-02 11:57:48 running bootstrap script ... ok
2023-11-02 11:57:49 performing post-bootstrap initialization ... ok
2023-11-02 11:57:49 syncing data to disk ... ok
2023-11-02 11:57:49
2023-11-02 11:57:49
2023-11-02 11:57:49 Success. You can now start the database server using:
2023-11-02 11:57:49
2023-11-02 11:57:49 pg_ctl -D /data/babelfish/ -l logfile start
2023-11-02 11:57:49
2023-11-02 11:57:49 waiting for server to start....2023-11-02 16:57:49.770 UTC [25] LOG: starting PostgreSQL 15.3 on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0, 64-bit
2023-11-02 11:57:49 2023-11-02 16:57:49.770 UTC [25] LOG: listening on IPv4 address "0.0.0.0", port 5432
2023-11-02 11:57:49 2023-11-02 16:57:49.770 UTC [25] LOG: listening on IPv6 address "::", port 5432
2023-11-02 11:57:49 2023-11-02 16:57:49.775 UTC [25] LOG: listening on Unix socket "/tmp/.s.PGSQL.5432"
2023-11-02 11:57:49 2023-11-02 16:57:49.777 UTC [25] LOG: listening on IPv4 address "0.0.0.0", port 1433
2023-11-02 11:57:49 2023-11-02 16:57:49.777 UTC [25] LOG: listening on IPv6 address "::", port 1433
2023-11-02 11:57:49 2023-11-02 16:57:49.781 UTC [28] LOG: database system was shut down at 2023-11-02 16:57:49 UTC
2023-11-02 11:57:49 2023-11-02 16:57:49.785 UTC [25] LOG: database system is ready to accept connections
2023-11-02 11:57:49 done
2023-11-02 11:57:49 server started
2023-11-02 11:57:49 CREATE ROLE
2023-11-02 11:57:49 DROP DATABASE
2023-11-02 11:57:49 CREATE DATABASE
2023-11-02 11:57:49 You are now connected to database "babelfish_db" as user "postgres".
2023-11-02 11:57:49 CREATE EXTENSION
2023-11-02 11:57:49 2023-11-02 16:57:49.980 UTC [34] WARNING: cast will be ignored because the source data type is a domain
2023-11-02 11:57:49 2023-11-02 16:57:49.985 UTC [34] WARNING: cast will be ignored because the source data type is a domain
2023-11-02 11:57:50 2023-11-02 16:57:50.007 UTC [34] WARNING: cast will be ignored because the source data type is a domain
2023-11-02 11:57:50 2023-11-02 16:57:50.012 UTC [34] WARNING: cast will be ignored because the source data type is a domain
2023-11-02 11:57:50 CREATE EXTENSION
2023-11-02 11:57:50 GRANT
2023-11-02 11:57:50 ALTER ROLE
2023-11-02 11:57:50 ALTER SYSTEM
2023-11-02 11:57:50 2023-11-02 16:57:50.758 UTC [25] LOG: received SIGHUP, reloading configuration files
2023-11-02 11:57:50 pg_reload_conf
2023-11-02 11:57:50 ----------------
2023-11-02 11:57:50 t
2023-11-02 11:57:50 (1 row)
2023-11-02 11:57:50
2023-11-02 11:57:50 2023-11-02 16:57:50.759 UTC [25] LOG: parameter "babelfishpg_tsql.database_name" changed to "babelfish_db"
2023-11-02 11:57:50 ALTER DATABASE
2023-11-02 11:57:50 2023-11-02 16:57:50.760 UTC [25] LOG: received SIGHUP, reloading configuration files
2023-11-02 11:57:50 pg_reload_conf
2023-11-02 11:57:50 ----------------
2023-11-02 11:57:50 t
2023-11-02 11:57:50 (1 row)
2023-11-02 11:57:50
2023-11-02 11:57:50 CALL
2023-11-02 11:57:50 waiting for server to shut down....2023-11-02 16:57:50.780 UTC [25] LOG: received fast shutdown request
2023-11-02 11:57:50 2023-11-02 16:57:50.783 UTC [25] LOG: aborting any active transactions
2023-11-02 11:57:50 2023-11-02 16:57:50.784 UTC [25] LOG: background worker "logical replication launcher" (PID 31) exited with exit code 1
2023-11-02 11:57:50 2023-11-02 16:57:50.786 UTC [26] LOG: shutting down
2023-11-02 11:57:50 2023-11-02 16:57:50.788 UTC [26] LOG: checkpoint starting: shutdown immediate
2023-11-02 11:57:50 2023-11-02 16:57:50.862 UTC [26] LOG: checkpoint complete: wrote 1438 buffers (8.8%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.016 s, sync=0.051 s, total=0.077 s; sync files=282, longest=0.004 s, average=0.001 s; distance=10136 kB, estimate=10136 kB
2023-11-02 11:57:50 2023-11-02 16:57:50.866 UTC [25] LOG: database system is shut down
2023-11-02 11:57:50 done
2023-11-02 11:57:50 server stopped
2023-11-02 11:57:49 initdb: warning: enabling "trust" authentication for local connections
2023-11-02 11:57:49 initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.
2023-11-02 11:57:49 NOTICE: database "babelfish_db" does not exist, skipping
2023-11-02 11:57:49 NOTICE: installing required extension "babelfishpg_tsql"
2023-11-02 11:57:49 NOTICE: installing required extension "uuid-ossp"
2023-11-02 11:57:49 NOTICE: installing required extension "babelfishpg_common"
2023-11-02 11:57:50 2023-11-02 16:57:50.905 UTC [36] LOG: starting PostgreSQL 15.3 on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0, 64-bit
2023-11-02 11:57:50 2023-11-02 16:57:50.905 UTC [36] LOG: listening on IPv4 address "0.0.0.0", port 5432
2023-11-02 11:57:50 2023-11-02 16:57:50.905 UTC [36] LOG: listening on IPv6 address "::", port 5432
2023-11-02 11:57:50 2023-11-02 16:57:50.910 UTC [36] LOG: listening on Unix socket "/tmp/.s.PGSQL.5432"
2023-11-02 11:57:50 2023-11-02 16:57:50.912 UTC [36] LOG: listening on IPv4 address "0.0.0.0", port 1433
2023-11-02 11:57:50 2023-11-02 16:57:50.912 UTC [36] LOG: listening on IPv6 address "::", port 1433
2023-11-02 11:57:50 2023-11-02 16:57:50.916 UTC [39] LOG: database system was shut down at 2023-11-02 16:57:50 UTC
2023-11-02 11:57:50 2023-11-02 16:57:50.942 UTC [36] LOG: database system is ready to accept connections
When I do:
ls /opt/babelfish/share/postgresql/extensionI get the following result:
adminpack--1.0--1.1.sql btree_gist--1.5--1.6.sql jsonb_plpython3u--1.0.sql pg_trgm.control
adminpack--1.0.sql btree_gist--1.6--1.7.sql jsonb_plpython3u.control pg_visibility--1.0--1.1.sql
adminpack--1.1--2.0.sql btree_gist.control lo--1.0--1.1.sql pg_visibility--1.1--1.2.sql
adminpack--2.0--2.1.sql citext--1.0--1.1.sql lo--1.1.sql pg_visibility--1.1.sql
adminpack.control citext--1.1--1.2.sql lo.control pg_visibility.control
amcheck--1.0--1.1.sql citext--1.2--1.3.sql ltree--1.0--1.1.sql pg_walinspect--1.0.sql
amcheck--1.0.sql citext--1.3--1.4.sql ltree--1.1--1.2.sql pg_walinspect.control
amcheck--1.1--1.2.sql citext--1.4--1.5.sql ltree--1.1.sql pgcrypto--1.0--1.1.sql
amcheck--1.2--1.3.sql citext--1.4.sql ltree.control pgcrypto--1.1--1.2.sql
amcheck.control citext--1.5--1.6.sql ltree_plpython3u--1.0.sql pgcrypto--1.2--1.3.sql
autoinc--1.0.sql citext.control ltree_plpython3u.control pgcrypto--1.3.sql
autoinc.control cube--1.0--1.1.sql moddatetime--1.0.sql pgcrypto.control
babelfishpg_common--1.0.0--1.1.0.sql cube--1.1--1.2.sql moddatetime.control pgrowlocks--1.0--1.1.sql
babelfishpg_common--1.0.0.sql cube--1.2--1.3.sql old_snapshot--1.0.sql pgrowlocks--1.1--1.2.sql
babelfishpg_common--1.1.0--1.2.0.sql cube--1.2.sql old_snapshot.control pgrowlocks--1.2.sql
babelfishpg_common--1.2.0--1.2.1.sql cube--1.3--1.4.sql pageinspect--1.0--1.1.sql pgrowlocks.control
babelfishpg_common--1.2.1--2.0.0.sql cube--1.4--1.5.sql pageinspect--1.1--1.2.sql pgstattuple--1.0--1.1.sql
babelfishpg_common--2.0.0--2.1.0.sql cube.control pageinspect--1.10--1.11.sql pgstattuple--1.1--1.2.sql
babelfishpg_common--2.1.0--2.2.0.sql dblink--1.0--1.1.sql pageinspect--1.2--1.3.sql pgstattuple--1.2--1.3.sql
babelfishpg_common--2.2.0--2.3.0.sql dblink--1.1--1.2.sql pageinspect--1.3--1.4.sql pgstattuple--1.3--1.4.sql
babelfishpg_common--2.3.0--2.4.0.sql dblink--1.2.sql pageinspect--1.4--1.5.sql pgstattuple--1.4--1.5.sql
babelfishpg_common--2.3.0--3.0.0.sql dblink.control pageinspect--1.5--1.6.sql pgstattuple--1.4.sql
babelfishpg_common--2.4.0--3.0.0.sql dict_int--1.0.sql pageinspect--1.5.sql pgstattuple.control
babelfishpg_common--3.0.0--3.1.0.sql dict_int.control pageinspect--1.6--1.7.sql plpgsql--1.0.sql
babelfishpg_common--3.1.0--3.2.0.sql dict_xsyn--1.0.sql pageinspect--1.7--1.8.sql plpgsql.control
babelfishpg_common--3.2.0.sql dict_xsyn.control pageinspect--1.8--1.9.sql plpython3u--1.0.sql
babelfishpg_common.control earthdistance--1.0--1.1.sql pageinspect--1.9--1.10.sql plpython3u.control
babelfishpg_money--1.1.0.sql earthdistance--1.1.sql pageinspect.control postgres_fdw--1.0--1.1.sql
babelfishpg_money.control earthdistance.control pg_buffercache--1.0--1.1.sql postgres_fdw--1.0.sql
babelfishpg_tds--1.0.0.sql file_fdw--1.0.sql pg_buffercache--1.1--1.2.sql postgres_fdw.control
babelfishpg_tds.control file_fdw.control pg_buffercache--1.2--1.3.sql refint--1.0.sql
babelfishpg_tsql--1.0.0--1.1.0.sql fixeddecimal--1.0.0--1.1.0.sql pg_buffercache--1.2.sql refint.control
babelfishpg_tsql--1.0.0.sql fuzzystrmatch--1.0--1.1.sql pg_buffercache.control seg--1.0--1.1.sql
babelfishpg_tsql--1.1.0--1.2.0.sql fuzzystrmatch--1.1.sql pg_freespacemap--1.0--1.1.sql seg--1.1--1.2.sql
babelfishpg_tsql--1.2.0--1.2.1.sql fuzzystrmatch.control pg_freespacemap--1.1--1.2.sql seg--1.1.sql
babelfishpg_tsql--1.2.1--1.3.0.sql hstore--1.1--1.2.sql pg_freespacemap--1.1.sql seg--1.2--1.3.sql
babelfishpg_tsql--1.2.1--2.0.0.sql hstore--1.2--1.3.sql pg_freespacemap.control seg--1.3--1.4.sql
babelfishpg_tsql--1.3.0--2.0.0.sql hstore--1.3--1.4.sql pg_prewarm--1.0--1.1.sql seg.control
babelfishpg_tsql--2.0.0--2.1.0.sql hstore--1.4--1.5.sql pg_prewarm--1.1--1.2.sql sslinfo--1.0--1.1.sql
babelfishpg_tsql--2.1.0--2.2.0.sql hstore--1.4.sql pg_prewarm--1.1.sql sslinfo--1.1--1.2.sql
babelfishpg_tsql--2.2.0--2.3.0.sql hstore--1.5--1.6.sql pg_prewarm.control sslinfo--1.2.sql
babelfishpg_tsql--2.3.0--2.4.0.sql hstore--1.6--1.7.sql pg_stat_statements--1.0--1.1.sql sslinfo.control
babelfishpg_tsql--2.3.0--3.0.0.sql hstore--1.7--1.8.sql pg_stat_statements--1.1--1.2.sql tablefunc--1.0.sql
babelfishpg_tsql--2.4.0--3.0.0.sql hstore.control pg_stat_statements--1.2--1.3.sql tablefunc.control
babelfishpg_tsql--2.5.0--3.0.0.sql hstore_plpython3u--1.0.sql pg_stat_statements--1.3--1.4.sql tcn--1.0.sql
babelfishpg_tsql--3.0.0--3.1.0.sql hstore_plpython3u.control pg_stat_statements--1.4--1.5.sql tcn.control
babelfishpg_tsql--3.1.0--3.2.0.sql insert_username--1.0.sql pg_stat_statements--1.4.sql tsm_system_rows--1.0.sql
babelfishpg_tsql--3.2.0.sql insert_username.control pg_stat_statements--1.5--1.6.sql tsm_system_rows.control
babelfishpg_tsql.control intagg--1.0--1.1.sql pg_stat_statements--1.6--1.7.sql tsm_system_time--1.0.sql
bloom--1.0.sql intagg--1.1.sql pg_stat_statements--1.7--1.8.sql tsm_system_time.control
bloom.control intagg.control pg_stat_statements--1.8--1.9.sql unaccent--1.0--1.1.sql
btree_gin--1.0--1.1.sql intarray--1.0--1.1.sql pg_stat_statements--1.9--1.10.sql unaccent--1.1.sql
btree_gin--1.0.sql intarray--1.1--1.2.sql pg_stat_statements.control unaccent.control
btree_gin--1.1--1.2.sql intarray--1.2--1.3.sql pg_surgery--1.0.sql uuid-ossp--1.0--1.1.sql
btree_gin--1.2--1.3.sql intarray--1.2.sql pg_surgery.control uuid-ossp--1.1.sql
btree_gin.control intarray--1.3--1.4.sql pg_trgm--1.0--1.1.sql uuid-ossp.control
btree_gist--1.0--1.1.sql intarray--1.4--1.5.sql pg_trgm--1.1--1.2.sql xml2--1.0--1.1.sql
btree_gist--1.1--1.2.sql intarray.control pg_trgm--1.2--1.3.sql xml2--1.1.sql
btree_gist--1.2--1.3.sql isn--1.0--1.1.sql pg_trgm--1.3--1.4.sql xml2.control
btree_gist--1.2.sql isn--1.1--1.2.sql pg_trgm--1.3.sql
btree_gist--1.3--1.4.sql isn--1.1.sql pg_trgm--1.4--1.5.sql
btree_gist--1.4--1.5.sql isn.control pg_trgm--1.5--1.6.sql
Which shows that plpython3u.control is there.
When I do:
ls /opt/babelfish/lib/postgresqlI get the following result:
_int.so citext.so hstore.so pg_buffercache.so plpython3.so utf8_and_euc_jp.so
adminpack.so cube.so hstore_plpython3.so pg_freespacemap.so postgres_fdw.so utf8_and_euc_kr.so
amcheck.so cyrillic_and_mic.so insert_username.so pg_prewarm.so refint.so utf8_and_euc_tw.so
auth_delay.so dblink.so isn.so pg_stat_statements.so seg.so utf8_and_gb18030.so
auto_explain.so dict_int.so jsonb_plpython3.so pg_surgery.so sslinfo.so utf8_and_gbk.so
autoinc.so dict_snowball.so latin2_and_win1250.so pg_trgm.so tablefunc.so utf8_and_iso8859.so
babelfishpg_common.so dict_xsyn.so latin_and_mic.so pg_visibility.so tcn.so utf8_and_iso8859_1.so
babelfishpg_money.so earthdistance.so libpqwalreceiver.so pg_walinspect.so test_decoding.so utf8_and_johab.so
babelfishpg_tds.so euc2004_sjis2004.so lo.so pgcrypto.so tsm_system_rows.so utf8_and_sjis.so
babelfishpg_tsql.so euc_cn_and_mic.so ltree.so pgoutput.so tsm_system_time.so utf8_and_sjis2004.so
basebackup_to_shell.so euc_jp_and_sjis.so ltree_plpython3.so pgrowlocks.so unaccent.so utf8_and_uhc.so
basic_archive.so euc_kr_and_mic.so moddatetime.so pgstattuple.so utf8_and_big5.so utf8_and_win.so
bloom.so euc_tw_and_big5.so old_snapshot.so pgxml.so utf8_and_cyrillic.so uuid-ossp.so
btree_gin.so file_fdw.so pageinspect.so pgxs utf8_and_euc2004.so
btree_gist.so fuzzystrmatch.so passwordcheck.so plpgsql.so utf8_and_euc_cn.so
Which shows that plpython3.so is there.
bachpro900 commented
Works perfectly! I did not use the right commit. My fault.
I am facing another problem, but this time when trying to install on Debian 11.
For information, below is the error. And I edited this line this way (RUN ./configure CFLAGS="-ggdb -fcommon") to resolve the problem.
c/tsql_analyze.o src/linked_servers.o src/pltsql_bulkcopy.o -L/opt/babelfish/lib -Wl,--as-needed -Wl,-rpath,'/opt/babelfish/lib',--enable-new-dtags -L/usr/local/lib -lantlr4-runtime -lcrypto
41.18 /usr/bin/ld: src/pl_comp-2.o:/workplace/BABEL_3_1_0__PG_15_2/contrib/babelfishpg_tsql/src/pl_comp-2.c:25: multiple definition of `pltsql_curr_compile_body_lineno'; src/pl_comp.o:/workplace/BABEL_3_1_0__PG_15_2/contrib/babelfishpg_tsql/src/pl_comp.c:62: first defined here
41.78 collect2: error: ld returned 1 exit status
41.78 make: *** [/opt/babelfish/lib/postgresql/pgxs/src/makefiles/../../src/Makefile.shlib:293: babelfishpg_tsql.so] Error 1
------
Dockerfile:109
--------------------
107 |
108 | WORKDIR ${PG_SRC}/contrib/babelfishpg_tsql
109 | >>> RUN make && make PG_CONFIG=${PG_CONFIG} install
110 |
111 | # Run stage
--------------------
ERROR: failed to solve: process "/bin/sh -c make && make PG_CONFIG=${PG_CONFIG} install" did not complete successfully: exit code: 2