Memory leak in S3 get_object?
Closed this issue · 2 comments
I've noticed a ballooning memory footprint in a worker of mine that uses the aws-sdk-core gem. I tracked the leak down to Aws::S3::Client.get_object
.
This is on ruby 2.2.0p0
and 2.0.23
of the gem.
I ran a simple script to see if this was the case.
require 'aws-sdk-core'
creds = Aws::Credentials.new("myaccesskey", "mysecretkey")
s3 = Aws::S3::Client.new(credentials: creds, region: 'us-east-1')
def mem(msg)
puts "#{msg} :: Memory #{`ps ax -o pid,rss | grep -E "^[[:space:]]*#{$$}"`
.strip.split.map(&:to_i)[1].to_s}KB"
end
1000.times.each do
mem("before")
s3.get_object(
bucket: "mybucket",
key: "mykey"
)
mem("after")
end
Here are the results of that script https://gist.github.com/abronte/f8a5a237175f91404a19. You can see a slow and steady increase in memory usage. This becomes pretty troublesome when you need to call get_object
millions of times per day.
This could just be a ruby 2.2 issue. I'm seeing better results with 2.0 (2.0.0p598) https://gist.github.com/abronte/ee88ef1a45587922ceb9
This seems to be the only thing I could find on somebody having issues with 2.2 memory in general http://stackoverflow.com/questions/27653207/does-ruby-2-2-have-memory-issues-on-heroku
Closing this since I don't think its specific to the gem. Manually calling GC.start
seems to help some.