HaxeFoundation/format

Avoid allocating memory for entire swf file in format.swf.Writer

tobil4sk opened this issue · 0 comments

Currently, when running format.swf.Writer, at the end, a memory block the size of the entire swf file is allocated:

format/format/swf/Writer.hx

Lines 1450 to 1452 in 318e49a

var bytes = o.getBytes();
var size = bytes.length;
if( compressed ) bytes = format.tools.Deflate.run(bytes);

This can cause issues with large swfs, for example: openfl/lime#1307.

A possible better solution is to:

  1. Work out the size of the swf in advance, so we don't need to wait until the large byte buffer has been filled to figure out the length for the header. Here is some example code from the haxe repo that does this: https://github.com/HaxeFoundation/haxe/blob/731dcd71f10c495a5a820449249fbb3d4b40a7c1/libs/swflib/swfParser.ml#L2062-L2066.
  2. When the byte buffer exceeds a certain size, send the current bytes to the compression stream (using Compress.execute). The compressed bytes can be written to the file, and we can continue with emptied buffers. This removes the need for a monolithic allocation and compression call at the end.