S3 server throws 403 error if Date Header for a S3GetObjectRequest is more than 15 minutes old.

[jasdeepsaini]

I am downloading a large amount of data (3.5 GB, ~1123 files and folders) from S3 to an iOS device. Like clockwork, the server will return a 403 error with no response body every 15 minutes. This is caused by the "Date" header of a S3GetObjectRequest being more than 15 minutes old.

I am using the transfer manager to download file as follows:

self.transferManager = [[S3TransferManager alloc] init];
self.transferManager.delegate = self;
self.transferManager.s3 = self.amazonS3Client;

[self.remoteFilesToDownload enumerateKeysAndObjectsUsingBlock:^(NSString *key, NSString *filePath, BOOL *stop) {
        [self.transferManager downloadFile:filePath bucket:self.bucketName key:key];
    }
}];

self.remoteFilesToDownload is a dictionary, where the keys are the keys from S3ObjectSummaries and the value is a filePath for where files will be saved.

When I run this code, the server will return a 403 status code for one of the downloads about 15 minutes after I start the download process.

I have traced the error to the S3Request class. The date property is being set once when the process begins, but the date header is not set until the actual download begins. My download process can take as long as 40 minutes. The transfer manager needs to support longer downloads.

If you want to reproduce the issue change the following S3Request method:

-(NSDate *)date
{
    if (_date == nil) {
        _date = [[NSDate date] retain];
    }
    return _date;
}

to

-(NSDate *)date
{
    if (_date == nil) {
        _date =[[[NSDate date] dateByAddingTimeInterval:16*60] retain];
    }
    return _date;
}

Every download will fail because the date header in the configureURLRequest method is more than 15 minutes old.

I noticed you don't seem to accept pull requests, so I am posting my solution below:

-(AmazonURLRequest *)configureURLRequest
{
    [super configureURLRequest];
    [self setHttpMethod:kHttpMethodGet];

    [self.urlRequest setValue:[NSString stringWithFormat:@"%lld", self.contentLength] forHTTPHeaderField:kHttpHdrContentLength];

    [self.urlRequest setValue:self.host forHTTPHeaderField:kHttpHdrHost];

    self.date = [NSDate date];
    [self.urlRequest setValue:[self.date stringWithRFC822Format] forHTTPHeaderField:kHttpHdrDate];

    if (nil != self.httpMethod) {
        [self.urlRequest setHTTPMethod:self.httpMethod];
    }
    if (nil != self.contentType) {
        [self.urlRequest setValue:self.contentType forHTTPHeaderField:kHttpHdrContentType];
    }
    if (nil != self.securityToken) {
        [self.urlRequest setValue:self.securityToken forHTTPHeaderField:kHttpHdrAmzSecurityToken];
    }

    [self.urlRequest setURL:self.url];

    return urlRequest;
}

Basically, I make sure the date header is initialized every time the configureURLRequest method is called instead of using the date value from when the Request was created.

[turacma]

Thanks a lot for the report.

We do in fact accept pull requests for bug fixes, so please go ahead and submit one so we can give you the appropriate credit.

[turacma]

Thanks a lot for the pull request. We've accepted it and will release as part of our next binary update.