Final Project: Install Rum
Opened this issue · 3 comments
Hi,
To create RUM, I first modified my Dockerfile in services/postgres directory:
lambda-server:~/bigdata/final_project $ cat services/postgres/Dockerf
ile
FROM postgres:13
# install system packages for building postgres extensions
RUN apt-get update && apt-get install -y \
less \
make \
vim \
postgresql-server-dev-13 \
postgresql-plpython3-13 \
python3 \
python3-pip \
sudo \
wget
# install a newer version of git
# these libraries are required just for installing a new version of git
RUN apt-get install -y \
libcurl4-gnutls-dev \
libexpat1-dev gettext \
libz-dev \
libssl-dev \
asciidoc \
xmlto \
docbook2x
RUN cd /tmp \
&& wget https://www.kernel.org/pub/software/scm/git/git-2.30.1.tar.gz \
&& tar -xzf git-2.30.1.tar.gz \
&& cd git-2.30.1 \
&& ./configure \
&& make \
&& make install \
&& rm -rf /tmp/git-2.30.1
# install postgres extensions from source
RUN cd /tmp \
&& git clone https://github.com/postgrespro/rum \
&& cd rum \
&& git checkout 1.3.7 \
&& make USE_PGXS=1 \
&& make USE_PGXS=1 install \
&& rm -rf /tmp/rum
WORKDIR /tmp/db
RUN mkdir /data && chown postgres /data
# copy over the pagila database;
# we rename the files so that they get executed in the correct order
COPY schema.sql /docker-entrypoint-initdb.d/01.sql
When I run :
docker-compose up -d --build
I get the following error:
Unpacking postgresql-client-13 (13.14-1.pgdg120+2) over (13.13-1.pgdg120+1) ...
dpkg: error processing archive /tmp/apt-dpkg-install-78gelh/136-postgresql-client-13_13.14-1.pgdg120+2_amd64.deb (--unpack):
cannot copy extracted data for './usr/lib/postgresql/13/bin/pg_isready' to '/usr/lib/postgresql/13/bin/pg_isready.dpkg-new': failed to write (Disk quota exceeded)
dpkg: error while cleaning up:
unable to restore backup version of '/usr/lib/postgresql/13/bin/pg_dumpall': Stale file handle
dpkg: error while cleaning up:
unable to restore backup version of '/usr/lib/postgresql/13/bin/pg_dump': Stale file handle
dpkg: error while cleaning up:
unable to restore backup version of '/usr/lib/postgresql/13/bin/pg_config': Stale file handle
dpkg: error while cleaning up:
unable to restore backup version of '/usr/lib/postgresql/13/bin/pg_basebackup': Stale file handle
dpkg: error while cleaning up:
unable to restore backup version of '/usr/lib/postgresql/13/bin/dropuser': Stale file handle
dpkg: error while cleaning up:
unable to restore backup version of '/usr/lib/postgresql/13/bin/dropdb': Stale file handle
dpkg: error while cleaning up:
unable to restore backup version of '/usr/lib/postgresql/13/bin/createuser': Stale file handle
dpkg: error while cleaning up:
unable to restore backup version of '/usr/lib/postgresql/13/bin/createdb': Stale file handle
dpkg: error while cleaning up:
unable to restore backup version of '/usr/lib/postgresql/13/bin/clusterdb': Stale file handle
tar: ./md5sums: Wrote only 9216 of 10240 bytes
tar: ./postinst: Cannot write: Disk quota exceeded
tar: ./postrm: Cannot write: Disk quota exceeded
tar: ./preinst: Cannot write: Disk quota exceeded
tar: ./prerm: Cannot write: Disk quota exceeded
tar: ./templates: Cannot write: Disk quota exceeded
tar: Exiting with failure status due to previous errors
dpkg-deb: error: tar subprocess returned error exit status 2
dpkg: error processing archive /tmp/apt-dpkg-install-78gelh/137-postgresql-13_13.14-1.pgdg120+2_amd64.deb (--unpack):
dpkg-deb --control subprocess returned error exit status 2
Selecting previously unselected package postgresql-plpython3-13.
Preparing to unpack .../138-postgresql-plpython3-13_13.14-1.pgdg120+2_amd64.deb ...
Unpacking postgresql-plpython3-13 (13.14-1.pgdg120+2) ...
dpkg: error processing archive /tmp/apt-dpkg-install-78gelh/138-postgresql-plpython3-13_13.14-1.pgdg120+2_amd64.deb (--unpack):
cannot copy extracted data for './usr/lib/postgresql/13/lib/bitcode/hstore_plpython3.index.bc' to '/usr/lib/postgresql/13/lib/bitcode/hstore_plpython3.index.bc.dpkg-new': failed to write (Disk quota exceeded)
tar: ./md5sums: Wrote only 4608 of 10240 bytes
tar: Exiting with failure status due to previous errors
dpkg-deb: error: tar subprocess returned error exit status 2
dpkg: error processing archive /tmp/apt-dpkg-install-78gelh/139-postgresql-server-dev-13_13.14-1.pgdg120+2_amd64.deb (--unpack):
dpkg-deb --control subprocess returned error exit status 2
Selecting previously unselected package publicsuffix.
Preparing to unpack .../140-publicsuffix_20230209.2326-1_all.deb ...
Unpacking publicsuffix (20230209.2326-1) ...
dpkg: error processing archive /tmp/apt-dpkg-install-78gelh/140-publicsuffix_20230209.2326-1_all.deb (--unpack):
cannot copy extracted data for './usr/share/doc/publicsuffix/copyright' to '/usr/share/doc/publicsuffix/copyright.dpkg-new': failed to write (Disk quota exceeded)
Selecting previously unselected package python3.11-dev.
Preparing to unpack .../141-python3.11-dev_3.11.2-6_amd64.deb ...
dpkg: unrecoverable fatal error, aborting:
unable to flush /var/lib/dpkg/updates/tmp.i after padding: Disk quota exceeded
E: Sub-process /usr/bin/dpkg returned an error code (2)
The command '/bin/sh -c apt-get update && apt-get install -y less make vim postgresql-server-dev-13 postgresql-plpython3-13 python3 python3-pip sudo wget' returned a non-zero code: 100
ERROR: Service 'postgres' failed to build : Build failed
I checked my disk quota:
lambda-server:~/bigdata/final_project $ df -h
Filesystem Size Used Avail Use% Mounted on
udev 126G 0 126G 0% /dev
tmpfs 26G 5.4M 26G 1% /run
/dev/nvme0n1p2 1.8T 1.4T 253G 85% /
tmpfs 126G 112K 126G 1% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 126G 0 126G 0% /sys/fs/cgroup
/dev/nvme0n1p1 511M 5.3M 506M 2% /boot/efi
tmpfs 26G 140K 26G 1% /run/user/1070
tmpfs 26G 0 26G 0% /run/user/1125
tmpfs 26G 0 26G 0% /run/user/1151
tmpfs 26G 0 26G 0% /run/user/1149
/dev/sda1 48T 30T 16T 67% /data
tmpfs 26G 0 26G 0% /run/user/1067
tmpfs 26G 12K 26G 1% /run/user/1284
tmpfs 26G 16K 26G 1% /run/user/1003
tmpfs 26G 164K 26G 1% /run/user/1240
tmpfs 26G 0 26G 0% /run/user/1306
tmpfs 26G 4.0K 26G 1% /run/user/1358
tmpfs 26G 368K 26G 1% /run/user/1323
tmpfs 26G 160K 26G 1% /run/user/1324
tmpfs 26G 160K 26G 1% /run/user/1347
tmpfs 26G 172K 26G 1% /run/user/1341
tmpfs 26G 160K 26G 1% /run/user/1319
tmpfs 26G 160K 26G 1% /run/user/1316
tmpfs 26G 300K 26G 1% /run/user/1328
tmpfs 26G 160K 26G 1% /run/user/1365
tmpfs 26G 160K 26G 1% /run/user/1336
tmpfs 26G 92K 26G 1% /run/user/1332
tmpfs 26G 368K 26G 1% /run/user/1346
tmpfs 26G 160K 26G 1% /run/user/1339
tmpfs 26G 168K 26G 1% /run/user/1366
tmpfs 26G 168K 26G 1% /run/user/1342
tmpfs 26G 24K 26G 1% /run/user/1322
tmpfs 26G 24K 26G 1% /run/user/1348
My UID is 1322, so I think I have 26 G available space. I am done with twitter_postgres_indexes. Should I delete pg_denormalized and pg_normalized_batch I created to get more space?
The df
command tells you how much disk space is free (df = disk free) on each drive. It is unrelated to quotas. du
will tell you how much disk space you are using (du = disk usage). Running du -hd1
in your home folder will find folders that are using too much space so you can delete them. You only have 10GB available, and the contents of the $HOME/bigdata
folder are not included in the quota.
I'm working on getting rum installed correctly and it seems like this is working. I'm curious, did you run into problems later on that necessitated the other installations?
FROM postgis/postgis
RUN apt-get update && apt-get install -y \
less \
make \
vim \
git \
gcc \
postgresql-server-dev-16
# install rum extensions from source
RUN cd /tmp \
&& pwd \
&& git clone https://github.com/postgrespro/rum \
&& pwd \
&& ls \
&& cd rum \
&& make USE_PGXS=1 \
&& make USE_PGXS=1 install \
&& rm -rf /tmp/rum
WORKDIR /tmp/db
RUN mkdir /data && chown postgres /data
# copy over the pagila database;
# we rename the files so that they get executed in the correct order
COPY schema.sql /docker-entrypoint-initdb.d/01.sql
FROM postgis/postgis
# Install system packages for building PostgreSQL extensions
RUN apt-get update && apt-get install -y \
git \
build-essential \
postgresql-server-dev-all \
postgresql-12-rum \
less \
make \
vim
RUN git clone https://github.com/postgrespro/rum /tmp/rum
WORKDIR /tmp/rum
RUN make USE_PGXS=1
RUN make USE_PGXS=1 install
# Install RUM extension for PostgreSQL
WORKDIR /tmp/db
RUN mkdir /data && chown postgres /data
# copy over the pagila database;
# we rename the files so that they get executed in the correct order
COPY schema.sql /docker-entrypoint-initdb.d/01.sql