Massive data allocation in _NSMutableGrowBytes

[federicocappelli]

I have a big memory management problem in the S3 framework, I have tried to download a big file with:

AmazonCredentials *credentials = [[AmazonCredentials alloc] initWithAccessKey: accessKey withSecretKey: secretKey];
    AmazonS3Client *connection = [[AmazonS3Client alloc] initWithCredentials: credentials];
    S3GetObjectRequest *downloadRequest = [[S3GetObjectRequest alloc] initWithKey:fileName withBucket: bucket];
    [downloadRequest setDelegate: self];
    [connection getObject: downloadRequest];

But the app crash with a memory warning in the didReceiveData: delegate

-(void)request:(AmazonServiceRequest *)request didReceiveData:(NSData *)data

the responsable appear to be a _NSMutableGrowBytes object

here the xcode allocation profile:

Why the framework appending this huge quantity of byte on this data structure?

[yosuke-matsuda]

Currently, S3GetObjectRequest loads the entire data into memory.

S3GetObjectRequest has a method called setRangeStart:rangeEnd:. You can use it to perform range gets and append the data to a local file. You can read about the range header specification here.

[federicocappelli]

Ok I have found the solution, probably the best practice is use:

NSOutputStream *stream = [[NSOutputStream alloc] initToFileAtPath:[filePath stringByAppendingPathComponent:fileName] append:NO];
    [stream scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
    [stream open];

    S3GetObjectRequest *request = [[S3GetObjectRequest alloc] initWithKey:fileName withBucket: bucket];
    request.outputStream = stream;
    request.delegate = self;
    [connection getObject:request];

and the delegate only for track progress and failures