maennchen/ZipStream-PHP

Update Docs for Flysystem 3

thedomeffm opened this issue · 4 comments

thedomeffm commented

Description

Great library thx.
I want to create a ZipStream from S3 to S3 with specific files from the Bucket.

I use Symfony with the Flysystem and AsyncAws (that does not support the "registerStreamWrapper" method).

I wanted to follow your example https://maennchen.dev/ZipStream-PHP/guide/FlySystem.html but the "putStream" method got removed see https://flysystem.thephpleague.com/docs/what-is-new/

It would be super cool if you could adapt the documentary (for Flysystem 3 compatibility).

Thank you

maennchen commented

@thedomeffm I‘m not using flysystem actively. Would you be able to open a PR woth the required doc changes?

NicolasCARPi commented

I guess it's simply a matter of changing putStream to writeStream.

thedomeffm commented

I've made a PR #265 for the code changes.

Probably you guys can me help out to improve the following line of codes.
I want to generate a zip which contains a lot of images and can get big (up to a few GB).
When I use fopen('php://memory', 'w+'); as stream, my weak server will break down.

As I said in the top comment I use the FlySystem3 with AsyncAws Adapter (that does not support the "registerStreamWrapper").
Is there a way that I can create a direct stream from my S3 Files -> ZipStream -> S3 Zip without exhausting my server RAM to much?

<?php 

// ...

$uploads = $this->getUploadsOfUser();

$tempStream = fopen('php://memory', 'w+');
$zipStream = new ZipStream(
    outputStream: $tempStream,
    enableZip64: true,
    outputName: 'test.zip',
);

foreach ($uploads as $upload) {
    $zipStream->addFileFromStream(
        fileName: $upload->originalUserFileName,
        stream: $uploadStorage->readStream($upload->getOriginalFilePath())
    );
}
$zipStream->finish();

$uploadStorage->writeStream('test.zip', $tempStream);
fclose($tempStream);
maennchen commented

@thedomeffm Thanks a lot for the PR ❤️

As for the other question: I believe your only other choice would be a temporary file.

The reason for that is that there seem to be only the write / writeStream functions and both most be created beforehand.

Ther might be a (complicated and hacky) solution using a custom stream that you could build using stream-wrapper-register:

  • You initialize a custom wrapper with a buffer.
  • On every read, you deliver from the buffer first and as soon as it is empty, you fill up the buffer again by adding the next file to the zip file and appending the content
  • When all files are added, return true for stream_eof

Using that approach, you should be able to reduce the maximum amount of memory needed to one file instead of all of them.

A better approach would be to propose a new method of writing a file to FlySystem where you get a writable resource from flysystem that could be passed as outputStream to ZipStream.

  • ❤️1