amazon-archives/aws-sdk-core-ruby

S3.get_object with block for not-exist-key writes out error xml

Closed this issue · 6 comments

takeru commented 
bucket = "valid-bucket"
key    = "not-exist-key"
filename = "./data.xxx"
File.unlink(filename) if File.exist?(filename)
File.open(filename,"wb") do |f|
  begin
    Aws::S3::Client.new.get_object(:bucket=>bucket, :key=>key) do |chunk|
      f.write(chunk)
    end
    puts "(no error)"
  rescue Object => e
    puts "e=#{e.inspect}"
  end
end
data = File.read(filename)
puts "data=[#{data}]"

exception raised:

#<NoMethodError: undefined method `rewind' for #<Seahorse::Client::BlockIO:0x007f88742b1548>>
ruby/2.1.0/gems/aws-sdk-core-2.0.22/lib/seahorse/client/http/response.rb:41:in `body_contents'

data.xxx includes error xml:

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Key>not-exist-key</Key><RequestId>..........</RequestId><HostId>.............</HostId></Error>
trevorrowe commented

Thanks for reporting this error and steps for reproduction. I've duplicated the error locally and I'll try to get a fix out with tomorrows release.

Generaly, I would not recomend using block form for reading. It has the downside that once bytes have been written to the block, the Ruby SDK is unable to retry failed requests. Might I recomend using the following form for get:

s3 = Aws::S3::Client.new
s3.get_object(bucket:'bucket-name, key:'key', response_target:'./path/to/file')

The fix involves updating the block form to not yield the response body when it is not a 200 response.

trevorrowe commented

The block should only receive actual data now, and the method will raise the error normally now. Thanks for reporting the issue!

takeru commented

Thanks!

But I am using:

obj = Aws::S3::Resource.new.bucket(@bucket).object(@key)
obj.get(:response_target=>"local_file")
sapientpants commented

Should this be treating any 2xx response code as success? I'm thinking specifically 206 as I've noticed that requests with a range specified leave the response in the body property of the get_object return value and don't get written out to the response_target with the 2.0.24 release. This used to work with 2.0.22.

trevorrowe commented

@marrrt Good catch. I've submitted a patch that expands this to handle 200..299 status codes.