Socket: Broken Pipe

[kylefleming]

I'm using this library to upload files from about 0.3MB-1MB. Every once in a while I get a "Socket: Broken Pipe" Failure. (I used charles proxy to see why it was not uploading.)

There's 2 issues I'm encountering. The first is that I'm getting this broken pipe issue at all. It's not that infrequent. I can usually queue up about 20 requests and have it happen. Looking at the failed vs succeeded requests, the only difference I see is that the connection is kept alive for these failed requests after several non-kept-alive requests, but it also has requests kept-alive that succeed. The keep alive stuff may be totally irrelevant though. If I retry the upload, it works fine. The failed upload is uploading to the same remote ip address that worked for the previous request only milliseconds before. All requests are done in serial.

The second issue is in how the sdk reports the issue. I encounter an immediate failure in the http response according to charles, but the aws sdk doesn't report an issue until it the timeout duration is up and gives the error message "Request timed out."

Do you have any suggestions on fixing these issues and is there any more information I can provide?

[Objective-Cloud]

+1 I have similar issues...

[turacma]

Apologies for the lack of contact on this issue. Can you clarify how you're using the library? Specifically:

1. Are you using the low level AmazonS3Client or S3TransferManager?
2. You indicated that you queue up requests and process them serially, how are you accomplishing this?
3. What region is your bucket in? Does this happen with other buckets?
4. Are these uploads done over Wifi or cellular?
5. What version of iOS are you targeting and what version is running on the device?

Given that we're relying on the behavior of NSURLConnection to handle connections we may need to follow up with Apple, but if we can narrow down the cause that will go a long way to being able to file a report if necessary.

[turacma]

Closing issue due to lack of follow up. Please feel free to reopen if you continue to have the issue.

[mat]

I ran into the same issue today and it looks like this type of error is happening to non-iOS S3 clients, too.

The article "Broken Pipe Error when using Boto + S3" at http://reterwebber.wordpress.com/2013/08/22/broken-pipe-error-when-using-boto-s3/ lead me to the solution that worked for me, too:

Tell the S3 library which region (host) you are using.

@turacma, to answer your questions:

1. Happened with both AmazonS3Client and S3TransferManager
2. eu-west-1 (ireland)
3. Over Wifi, haven't tested otherwise.
4. Simulator running iOS 7.0.4, built against SDK 7.

Incidentally I ran into the same issue last week when using http://s3tools.org/s3cmd (which is written in python) while writing into my Ireland located bucket. The symptom (Broken Pipe) was the same, the solution (explicit region host selection) as well.

Here is the code I used:

- (void) uploadFileToS3
{
//    NSString* hostname = @"s3.amazonaws.com"; // Triggered "Broken Pipe" error!
    NSString* hostname = @"s3-eu-west-1.amazonaws.com";
    NSString* bucketName = @"iosbucket";

    NSString* accessKey = @"XXX";
    NSString* secretKey = @"YYY";

    NSString* localImageName = @"galaxy.jpg"; // ~15MB
    NSString* keyName = @"galaxy.jpg";

    AmazonS3Client* s3 = [[AmazonS3Client alloc] initWithAccessKey:accessKey withSecretKey:secretKey];

    S3PutObjectRequest* por = [[S3PutObjectRequest alloc] initWithKey:keyName inBucket:bucketName];
    por.data = UIImageJPEGRepresentation([UIImage imageNamed:localImageName], 0.8);;
    por.contentType = @"image/jpeg";
    por.hostName = hostname;

    BOOL useTransferManager = YES;
    if (useTransferManager)
    {
        S3TransferManager* tm = [S3TransferManager new];
        tm.s3 = s3;
        [tm upload:por];
    }
    else
    {
        [s3 putObject:por];
    }
}