Why is the result not identical to the official utility?
Agagamand opened this issue · 4 comments
If you create a .tar.zst using the official utility, the the result will contain more information (Uncompressed | Ratio | Check) than if you create .tar.zst with ZstdSharp. Is there any way to fix it?
string srcFilePath = "R:\\file.dat";
using FileStream outputStream = new("R:\\ZstdSharp.tar.zst", FileMode.CreateNew, FileAccess.Write);
using CompressionStream zstdStream = new(outputStream, 22);
zstdStream.SetParameter(ZSTD_cParameter.ZSTD_c_nbWorkers, Environment.ProcessorCount);
using TarWriter tarStream = new(zstdStream, TarEntryFormat.Pax, leaveOpen: false);
using FileStream srcStream = new(srcFilePath, FileMode.Open, FileAccess.Read);
{
PaxTarEntry tarEntry = new(TarEntryType.RegularFile, srcFilePath)
{
DataStream = srcStream,
ModificationTime = File.GetLastWriteTime(srcFilePath)
};
tarStream.WriteEntry(tarEntry);
}
You need to set the same parameters that were used for compression by the zstd utility.
- To add checksum use:
zstdStream.SetParameter(ZSTD_cParameter.ZSTD_c_checksumFlag, 1);
- To add Uncompressed/Ratio you need to pass total input data size to be compressed by stream API:
There is aZSTD_CCtx_setPledgedSrcSize
function for this, but it is not available from the safe API. I will add it.
2. To add Uncompressed/Ratio you need to pass total input data size to be compressed by stream API
Can ZstdSharp calculate the amount of incoming data itself?
I just don't I don't understand how I can calculate it myself in the example above, where incoming data is simultaneously incoming data are created (packed in .tar) and sent as a stream to ZstdSharp at once.
Can ZstdSharp calculate the amount of incoming data itself?
It needs to be written in the frame header, so you need to know in advance.
I just don't I don't understand how I can calculate it myself in the example above, where incoming data is simultaneously incoming data are created (packed in .tar) and sent as a stream to ZstdSharp at once.
If we know in advance the size of the files to be packed, it should be possible. But I'm not sure if the .NET Tar API allows to calculate it.
@Agagamand
Added SetPledgedSrcSize
in master