Is it possible to unpack while downloading for erigon snapshot?

[lilcryptopump]

Is it possible to unpack the archive while downloading?

I have limited space and cant download all 7 parts, concatenate them in one file and then extract it. I have 10 TB and it seams not will be enough.

Can I unpack each archive while downloading it?
Something like this will work?

wget -q -O - https://[url]/erigon_data_20231130.lz4.000 | lz4 -d -c > mdbx.dat
wget -q -O - https://[url]/erigon_data_20231130.lz4.001 | lz4 -d -c > mdbx.dat
...
wget -q -O - https://[url]/erigon_data_20231130.lz4.007 | lz4 -d -c > mdbx.dat

[blxdyx]

It won't work . At least need 12T.

[du5]

Is it possible to unpack the archive while downloading?

I have limited space and cant download all 7 parts, concatenate them in one file and then extract it. I have 10 TB and it seams not will be enough.

Can I unpack each archive while downloading it? Something like this will work?

It can be done if it is cut before packing

[blxdyx]

Is it possible to unpack the archive while downloading?

I have limited space and cant download all 7 parts, concatenate them in one file and then extract it. I have 10 TB and it seams not will be enough.
Can I unpack each archive while downloading it? Something like this will work?

It can be done if it is cut before packing

Thanks, will try next time.

[du5]

This should complete the download via

wget url0 | lz4 -d -c > mdbx.dat 
wget url1 | lz4 -d -c >> mdbx.dat
...
wget urln | lz4 -d -c >> mdbx.dat
# checksum 
openssl sha256 mdbx.dat

[du5]

[du5]

I think checksum must be completed. Using pipes to download, decompress and append files is a very unsafe operation. But it can reduce huge disk space requirements

[lilcryptopump]

Thanks, will try next time.

Hello @blxdyx, I would like to know if we will be able to get this feature? 🙏