latest dpkg-deb zstd compression is not supported by reprepro

Question

latest dpkg-deb zstd compression is not supported by reprepro

jean-christophe-manciot opened this issue 3 years ago · 15 comments

jean-christophe-manciot commented 3 years ago

Ubuntu impish
reprepro: 5.3.0-1.1 multiple-versions (630bb1e)
dpkg: 1.20.9ubuntu2

Ubuntu has switched default dpkg-deb compression from xz to zstd: cf. changelog.
Adding a deb package built using the latest dpkg-deb to a repository managed by reprepro now leads to:

reprepro --component stable -Vb . includedeb impish ../../linux-buildinfo-5.13.0-12-generic_5.13.0-12.12_amd64.deb
Could not find a suitable control.tar file within '../../linux-buildinfo-5.13.0-12-generic_5.13.0-12.12_amd64.deb'!
There have been errors!

There is no such issue when adding packages built with dpkg 1.20.9.
Unfortunately, downgrading dpkg to the latter prevents any further installation of binary packages built and distributed by Ubuntu.

Answer 1 · 2021-07-20T09:20:12.000Z

There has been a recent change also to reprepro in debian to add support for Zstd compressed debs: cf. changelog.

Answer 2 · 2021-07-20T14:47:02.000Z

I rebased the multiple-versions-debian on the Debian package version 5.3.0-1.3 to pick up zstd support and verified that the shunit2 tests still pass. Please test.

Answer 3 · 2021-07-20T20:01:20.000Z

I've built 3908caf using latest dpkg-deb on Ubuntu impish.

So far, multiple versions reprepro 5.3.0-1.4 works like a charm as can be seen on this reprepro deb repository.

I'd like to test it for a couple of complete build sessions before closing this issue.

Answer 4 · 2021-07-23T08:31:49.000Z

3908caf has passed the tests on:

Ubuntu 21.10 impish with dpkg-deb 1.20.9ubuntu2
Debian 11.0 bullseye with dpkg-deb 1.20.9

Answer 5 · 2021-08-05T14:05:51.000Z

Actually, I've just happened to see the following error message:

reprepro --component stable -Vb . includedeb impish gns3-gui_2.2.23-21.10_amd64.deb
zstd: error 70 : Write error : cannot write decoded block : Broken pipe

Maybe it's worth noting that the deb file is signed.

Answer 6 · 2021-12-13T08:17:37.000Z

Actually, I've just happened to see the following error message:
reprepro --component stable -Vb . includedeb impish gns3-gui_2.2.23-21.10_amd64.deb
zstd: error 70 : Write error : cannot write decoded block : Broken pipe
Maybe it's worth noting that the deb file is signed.

Just ran into this exact problem on 5.3.0 (stable on Ubuntu 20.04), which means reprepro is unusable. Are there any news on this?

Answer 7 · 2021-12-13T14:25:26.000Z

Just ran into this exact problem on 5.3.0 (stable on Ubuntu 20.04), which means reprepro is unusable. Are there any news on this?

I wouldn't say 'reprepro is unusable': despite the previous error message showing up from time to time:

the affected binary packages are correctly inserted into the repository; for instance, one affected package is available online within the repository
the reprepro repository is running correctly

# Checking if all packages in Ubuntu impish distribution have all files needed properly registered
reprepro -Vb . check impish
Checking impish...

# Checking if all files believed to be in pool are actually still there and have the known md5sum, sha1 & sha256
reprepro -Vb . checkpool

Answer 8 · 2021-12-13T17:28:30.000Z

I wouldn't say 'reprepro is unusable': despite the previous error message showing up from time to time:

* the affected binary packages are correctly inserted into the repository; for instance, the previously affected package is [available online within the repository](https://git.sdxlive.com/PPA/commit/?id=b85bbd5c65ed56fda70ac31db52e0589e34d0587)

* the reprepro repository is running correctly

oh, amazing. I didn't realize they were imported. I ran into the problem while doing reprepro update and it just spammed the log, so I assumed it wasn't syncing properly.
Thanks!

Answer 9 · 2022-02-19T00:44:54.000Z

Same problem here.

ubuntu-21.10 reprepro -b /var/www/html/ubuntu-21.10 -C main includedeb impish /root/kernel/linux-*.deb
Could not find a suitable control.tar file within '/root/kernel/linux-headers-5.17.0-rc4-ubuntu-starfive-rc4_5.17.0-rc4-ubuntu-starfive-rc4-1_riscv64.deb'!
Could not find a suitable control.tar file within '/root/kernel/linux-image-5.17.0-rc4-ubuntu-starfive-rc4_5.17.0-rc4-ubuntu-starfive-rc4-1_riscv64.deb'!
Could not find a suitable control.tar file within '/root/kernel/linux-image-5.17.0-rc4-ubuntu-starfive-rc4-dbg_5.17.0-rc4-ubuntu-starfive-rc4-1_riscv64.deb'!
Could not find a suitable control.tar file within '/root/kernel/linux-libc-dev_5.17.0-rc4-ubuntu-starfive-rc4-1_riscv64.deb'!
There have been errors!

Answer 10 · 2022-04-27T09:02:58.000Z

I've been getting this same error with packages build with ubuntu 22.04. I run reprepro in docker so it's pretty reproducible.

Here's how I'm building reprepro (where reprepro-multiple-versions-debian.zip is the zip download from github of the multiple-versions-debian branch of this repository):

FROM ubuntu:22.04
ADD reprepro-multiple-versions-debian.zip /tmp/

RUN \
	apt-get --quiet --quiet update && \
	apt-get install -y \
		curl \
		nano \
		jq \
		gcc \
                make \
		libarchive-dev  \
		libbz2-dev  \
		libdb-dev \
		libgpgme-dev \
		liblzma-dev \
		libzstd-dev \
		unzip \
		zlib1g-dev
RUN unzip /tmp/reprepro-multiple-versions-debian.zip -d /tmp && \
	cd /tmp/reprepro-multiple-versions-debian && \
	/tmp/reprepro-multiple-versions-debian/configure && \
	make install -C /tmp/reprepro-multiple-versions-debian

I redownloaded the zip yesterday and the source does contain changes mentioning zstd, so it's still an issue. Am I perhaps missing some configuration flags required to support zstd packaged debs?

Answer 11 · 2022-04-27T09:05:00.000Z

Can you try installing the zstd package?

Answer 12 · 2022-04-27T09:12:36.000Z

@bdrung oh that fixed it, thank you!

Answer 13 · 2022-05-04T21:32:20.000Z

I am not be able to get reprepro running. I test with the @jean-christophe-manciot but I get the same message "broken pipe" :-(

Answer 14 · 2022-08-17T12:25:52.000Z

i build on Debian SID, and test with a local repo *( reprepro 5.3.1-1)

Then I import on a Debian Stretch server reprepro 5.3.0-1~bpo9+1 , and i get. :
Could not find a suitable control.tar file within

And importing it works fine on the debian SID/Unstable version

To be added, importing the DSC works fine, only the multichanges or changes file give above error.

zstd and libzstd1 ( 1.3.8+dfsg-3~bpo9+1) is installed on the stretch server.

i'll see if in can backport the debian sid version to stretch. i'll report it back

Answer 15 · 2022-08-17T13:14:33.000Z

i build on Debian SID, and test with a local repo *( reprepro 5.3.1-1)

Then I import on a Debian Stretch server reprepro 5.3.0-1~bpo9+1 , and i get. : Could not find a suitable control.tar file within

And importing it works fine on the debian SID/Unstable version

To be added, importing the DSC works fine, only the multichanges or changes file give above error.

zstd and libzstd1 ( 1.3.8+dfsg-3~bpo9+1) is installed on the stretch server.

i'll see if in can backport the debian sid version to stretch. i'll report it back

i rebuilded reprepro from debian experimental ( 5.4.0 ) and zstd (1.5.2+dfsg-1) from debian Sid to Debian Stretch.
Now it works

what i used can be found here. https://downloads.van-belle.nl/diverse/
deb and sources included in the tar.gz.

Please note..
This breaks broke the index (db) files of reprepro, so im rebuilding but with above versions.

Importing shows : zstd: /stdout: Broken pipe

but so far it all works.