Postgres setup fails during installation.
chrisnyc opened this issue · 10 comments
chrisnyc commented
Hi, so installing v21 on Ubuntu 20.04 on GCP.
Seems the postgresql setup failed. I think this is because I'm using a low-spec e2-medium VM in GCP and with everything else going on in the script, the machine is bogged down at this point. Below are the logs showing the failures:
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
Success. You can now start the database server using:
pg_ctlcluster 12 main start
Ver Cluster Port Status Owner Data directory Log file
12 main 5432 down postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log
update-alternatives: using /usr/share/postgresql/12/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode
Setting up sysstat (12.2.0-2ubuntu0.1) ...
..........
Setting up postgresql-server-dev-12 (12.9-0ubuntu0.20.04.1) ...
Setting up postgresql-server-dev-all (214ubuntu0.1) ...
Processing triggers for systemd (245.4-4ubuntu3.16) ...
Processing triggers for libc-bin (2.31-0ubuntu9.7) ...
createuser: error: could not connect to database template1: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
createdb: error: could not connect to database template1: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
psql: error: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
psql: error: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
psql: error: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
psql: error: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
Synchronizing state of postgresql.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable postgresql
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 137 100 137 0 0 2107 0 --:--:-- --:--:-- --:--:-- 2107
100 291k 0 291k 0 0 2010k 0 --:--:-- --:--:-- --:--:-- 2010k
gvm-libs-21.4.4/
gvm-libs-21.4.4/.clang-format
gvm-libs-21.4.4/.codecov.yml
.........
UUID=$(/opt/gvm/sbin/gvmd --get-scanners | grep Default | awk '{print $1}')
sleep 10
echo
/opt/gvm/sbin/gvmd --modify-scanner=$UUID --scanner-host=/opt/gvm/var/run/ospd.sock
sleep 10
/opt/gvm/sbin/gvmd --verify-scanner=$UUID
/opt/gvm/sbin/gvmd --create-user gvmadmin --password=StrongPass
UUID=$(/opt/gvm/sbin/gvmd --get-users --verbose | grep admin | awk '{print $2}')
/opt/gvm/sbin/gvmd --modify-setting 78eceaec-3385-11ea-b237-28d24461215b --value $UUID
(gvmd:21293): md main-CRITICAL **: 11:38:55.777: gvmd: g_option_context_parse: Missing argument for --value
chrisnyc commented
I solved the problem by manually installing postgresql first with the following commands.
Note that I added the postgresql-client. I'm not sure if that helped or not.
apt-get -y install postgresql postgresql-contrib postgresql-server-dev-all postgresql-client
systemctl enable postgresql
systemctl start postgresql
Next, I waited a minute, and then ran
systemctl status postgresql
The output should look like this:
● postgresql.service - PostgreSQL RDBMS
Loaded: loaded (/lib/systemd/system/postgresql.service; enabled; vendor preset: enabled)
Active: active (exited) since Fri 2022-04-15 16:09:22 EDT; 25s ago
Main PID: 13944 (code=exited, status=0/SUCCESS)
Tasks: 0 (limit: 4696)
Memory: 0B
CGroup: /system.slice/postgresql.service
Apr 15 16:09:22 dev-int-ovas systemd[1]: Starting PostgreSQL RDBMS...
Apr 15 16:09:22 dev-int-ovas systemd[1]: Finished PostgreSQL RDBMS.
Then I commented out the installation steps on the lines 112,113 and 120,121 of install_gvm.sh like this:
#apt-get -y install postgresql postgresql-contrib postgresql-server-dev-all
#systemctl restart postgresql
sudo -Hiu postgres createuser gvm
sudo -Hiu postgres createdb -O gvm gvmd
sudo -Hiu postgres psql -c 'create role dba with superuser noinherit;' gvmd
sudo -Hiu postgres psql -c 'grant dba to gvm;' gvmd
sudo -Hiu postgres psql -c 'create extension "uuid-ossp";' gvmd
sudo -Hiu postgres psql -c 'create extension "pgcrypto";' gvmd
#systemctl restart postgresql
#systemctl enable postgresql
Finally, I ran
sudo ./install_gvm.sh -v 21 -u
per the regular procedure.
The output of the postgres commands above should look like this:
CREATE ROLE
GRANT ROLE
CREATE EXTENSION
CREATE EXTENSION
Thanks very much for this script! It's very helpful!
yu210148 commented
Hi there,
Thanks for this. Ya, it could be some sort of timing issue. I haven't
noticed this in testing but that could be the specs of the machine. Next
time I have it open I'll add in the postgresql-client to the list of things
that get installed with apt just in case. I don't think it would do any
harm to have it in there.
Kev.
…On Fri, Apr 15, 2022, 4:34 PM chrisnyc ***@***.***> wrote:
I think this may be occurring because I'm using a lowish-spec e2-medium VM
in GCP.
I solved the problem by manually installing postgresql first with the
following commands.
Note that I added the postgresql-client. I'm not sure if that helped or
not.
apt-get -y install postgresql postgresql-contrib postgresql-server-dev-all postgresql-client
systemctl enable postgresql
systemctl start postgresql
Next, I waited a minute, and then ran
systemctl status postgresql
The output should look like this:
● postgresql.service - PostgreSQL RDBMS
Loaded: loaded (/lib/systemd/system/postgresql.service; enabled; vendor preset: enabled)
Active: active (exited) since Fri 2022-04-15 16:09:22 EDT; 25s ago
Main PID: 13944 (code=exited, status=0/SUCCESS)
Tasks: 0 (limit: 4696)
Memory: 0B
CGroup: /system.slice/postgresql.service
Apr 15 16:09:22 dev-int-ovas systemd[1]: Starting PostgreSQL RDBMS...
Apr 15 16:09:22 dev-int-ovas systemd[1]: Finished PostgreSQL RDBMS.
Then I commented out the installation steps on the lines 112,113 and
120,121 of install_gvm.sh like this:
#apt-get -y install postgresql postgresql-contrib postgresql-server-dev-all
#systemctl restart postgresql
sudo -Hiu postgres createuser gvm
sudo -Hiu postgres createdb -O gvm gvmd
sudo -Hiu postgres psql -c 'create role dba with superuser noinherit;' gvmd
sudo -Hiu postgres psql -c 'grant dba to gvm;' gvmd
sudo -Hiu postgres psql -c 'create extension "uuid-ossp";' gvmd
sudo -Hiu postgres psql -c 'create extension "pgcrypto";' gvmd
#systemctl restart postgresql
#systemctl enable postgresql
Finally, I ran
sudo ./install_gvm.sh -v 21 -u
per the regular procedure.
The output of the postgres commands above should look like this:
CREATE ROLE
GRANT ROLE
CREATE EXTENSION
CREATE EXTENSION
Thanks very much for this script! It's very helpful!
—
Reply to this email directly, view it on GitHub
<#73 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAKFY4NMIQPJKXWL5TDHCULVFHHDVANCNFSM5TQWZ4UQ>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
chrisnyc commented
I think trying to install Postgresql while the build is running in the background may be causing the issue.
Running the postgresql installation before you kickoff the build would be one way to solve it with some certainty.
Or possibly, a couple minutes of sleep after the startup of postgresql on line 113, just before the call to "createuser gvm" might give enough time for postgresql to come up before trying to "createuser gvm" would be a simple way to resolve it.
Thanks again for putting this together. Saves me lots of work!
yu210148 commented
Ya, I could throw a sleep statement in there. Will do next time I have it
open or if you're looking at it if you could send me a pull request I'll
merge it in. I'm not at my screen at the moment.
Kev.
…On Fri, Apr 15, 2022, 6:21 PM chrisnyc ***@***.***> wrote:
I think trying to install Postgresql while the build is running in the
background may be causing the issue.
Running the postgresql installation before you kickoff the build would be
one way to solve it with some certainty.
Or possibly, a couple minutes of sleep after the startup of postgresql on
line 113, just before the call to "createuser gvm" might give enough time
for postgresql to come up before trying to "createuser gvm" would be a
simple way to resolve it.
Thanks again for putting this together. Saves me lots of work!
—
Reply to this email directly, view it on GitHub
<#73 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAKFY4JFELFMQXXSWKPMVR3VFHTX3ANCNFSM5TQWZ4UQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
chrisnyc commented
I'll try a few things out and if I find a solid solution I'll send you a pull request. Thanks again!
yu210148 commented
Cool, thanks 👍
Kev.
…On Fri, Apr 15, 2022, 6:35 PM chrisnyc ***@***.***> wrote:
I'll try a few things out and if I find a solid solution I'll send you a
pull request. Thanks again!
—
Reply to this email directly, view it on GitHub
<#73 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAKFY4MAHNH35TDMRMAUIRDVFHVLVANCNFSM5TQWZ4UQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
yu210148 commented
Hi again,
I just added a sleep 60 statement to allow postgres to restart after line 113. Ya, it might be annoying to someone not having this issue but the script already takes a long time to do what it's doing so what's 60 more seconds :)
kev.
chrisnyc commented
Hi, thanks for trying the "sleep 60" after the postgresql install, but it seems waiting alone does not solve the issue. I tested it and here's the result:
Setting up postgresql-server-dev-all (214ubuntu0.1) ...
Processing triggers for systemd (245.4-4ubuntu3.16) ...
Processing triggers for libc-bin (2.31-0ubuntu9.7) ...
Sleeping for a minute to let postgres restart.
https://github.com/yu210148/gvm_install/issues/73 has more info on why we're waiting here.
createuser: error: could not connect to database template1: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
createdb: error: could not connect to database template1: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
psql: error: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
psql: error: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
psql: error: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
psql: error: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
Synchronizing state of postgresql.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable postgresql
I will a few more things and let you know what I find. Thanks again for your help.
chrisnyc commented
I'm stumped. I've tried everything I can think of to get Postgres to work immediately after installing it within the script.
If I install Postgres in my terraform startup script that runs when the instance is first created, everything works fine.
The problem has something to do with the installation of Postgres, and the initialization of the database for first use.
It's possible that I'm seeing this because of CIS server hardening I've got in place, where they require you do things like set /var/log to 'noexec'.
The simple work-around is to just install postgres before you run the script.
yu210148 commented
Odd, okay, thanks for looking into it and documenting a workaround here.
Hopefully, anyone else running into the issue will find this thread.
Kev.
…On Sun, Apr 17, 2022, 10:30 AM chrisnyc ***@***.***> wrote:
I'm stumped. I've tried everything to get Postgres to work immediately
after installing it within the script.
If I install Postgres in my terraform startup script that runs when the
instance is first created, everything works fine.
The problem has something to do with the installation of Postgres, and the
initialization of the database for first use.
It's possible that I'm seeing this because of CIS server hardening I've
got in place, where they require you do things like set /var/log to
'noexec'.
The simple work-around is to just install postgres before you run the
script.
—
Reply to this email directly, view it on GitHub
<#73 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAKFY4ME3KMLKLRZFH2OMRTVFQN77ANCNFSM5TQWZ4UQ>
.
You are receiving this because you commented.Message ID:
***@***.***>