Corrupted images inside archive with more than 700 images
Niremizov opened this issue · 1 comments
Description of the problem
Creating archive with more than ~ 700 files, each 600Kb - 3074 Kb size , leads to few corrupted images. Corrupted images are not the same if several archivation performed, so this is some kind of floating error.
Tried to investigate, appeared that due to turned off deflate no more corruptions appeared. Notice: deflate level -1 do not help, only change on method helped.
Example code
$s3Client->registerStreamWrapper();
$fileName = $context['results']['filename'];
$path = "s3://{$bucket}/tmp/{$fileName}";
$filepath = file_directory_temp() . '/' . $fileName;
$local_zip = new ZipArchive;
$local_zip->open($filepath, ZipArchive::CREATE);
$streamOutput = fopen($filepath, 'w', FALSE);
$zip= new ZipStream($path);
$zip->opt->setOutputStream($streamOutput);
$streamContext = stream_context_create(
[
's3' => ['seekable' => TRUE],
]
);
foreach ($imageProperties as $imageProperty) {
$filePath = "s3://$imageProperty->fileName";
$streamRead = fopen($filePath, 'r', FALSE, $streamContext);
if ($streamRead !== FALSE) {
$options = new \ZipStream\Option\File();
$options->setSize(filesize($filePath));
// Turned off deflate
$options->setMethod(\ZipStream\Option\Method::STORE());
$zip->addFileFromStream($imageProperty->fileName, $streamRead, $options);
}
}
$zip->finish();
$s3Client->putObject(
[
'Bucket' => $bucket,
'Key' => "{$this->tempDir}/{$fileName}",
'Body' => fopen($filepath, 'r'),
'ACL' => 'public-read',
]
);
}
Informations
- ZipStream-PHP version: 1.2
- PHP version: 7.1.33
As a result
- Probably it deserves mentioning in Wiki:
- image with size more than 600 Kb - are large, and probably should be processed with STORE method. With deflate mode can be corrupted.
- deflate - can corrupt files
- setDeflateLevel(-1); - does not turn of deflate, files still would be processed through deflate functions, but won't be compressed.
- Found that deflate encoding could be be changed, and it is set as ZLIB_ENCODING_RAW - and this constant for some reason is not described on zlib site http://www.zlib.net/manual.html. Why does exactly this encoding set?
I have tried to reproduce this issue and have failed to do so.
Since the issue is relatively old and the issue was related to deflation, i assume that it had something to do with zlib itself.
I will therefore close this issue for now. Should someone still observe the described behaviour, please reopen the issue and provide details about both the zipstream version, your PHP version and the operating system that was used.