Ruby 1.9 compatibility - use bytesize instead of size
Closed this issue · 15 comments
memcache-client 1.8.5 running on Ruby 1.9 can't cope with non-ascii utf-8 strings as data ("values"). The memcached protocol requires the number of bytes, but memcache-client passes on number of characters:
it is using value.to_s.size
everywhere, with Ruby 1.9 it should be value.to_s.bytesize
.
Feel free to fork, fix and send pull request. Otherwise, I'll get to it one of these days.
Seems like this has been fixed :)
Yep, sorry forgot to mention here. Issue should be fixed in HEAD.
Sorry, false positive. Looks like using Ruby 1.9.1 p378 and Rails 2.3.8, I am still getting the Encoding::CompatibilityError exception when attempting to send UTF-8 values to the memcache-client.
>> Rails.cache.fetch(symbol) { first(:conditions => { :symbol => "£" }) }
Currency Load (0.3ms) SELECT * FROM `currencies` WHERE (`currencies`.`symbol` = '£') LIMIT 1
Encoding::CompatibilityError: incompatible character encodings: UTF-8 and ASCII-8BIT
from /Users/jdelsman/Ultraspeed/core/vendor/plugins/memcache-client/lib/memcache.rb:374:in `block in set'
from /Users/jdelsman/Ultraspeed/core/vendor/plugins/memcache-client/lib/memcache.rb:892:in `with_server'
from /Users/jdelsman/Ultraspeed/core/vendor/plugins/memcache-client/lib/memcache.rb:367:in `set'
from /Users/jdelsman/.rvm/gems/ruby-1.9.1-p378/gems/activesupport-2.3.8/lib/active_support/cache/mem_cache_store.rb:82:in `write'
from /Users/jdelsman/.rvm/gems/ruby-1.9.1-p378/gems/activesupport-2.3.8/lib/active_support/cache/strategy/local_cache.rb:51:in `write'
from /Users/jdelsman/.rvm/gems/ruby-1.9.1-p378/gems/activesupport-2.3.8/lib/active_support/cache.rb:165:in `fetch'
Try putting:
# encoding: utf-8
at the top of the source file which has the non-ascii string.
Already had that. I applied the following money patch, and now I'm getting ASCII back:
http://gnuu.org/2009/11/06/ruby19-rails-mysql-utf8/
ruby-1.9.1-p378 > Currency.sterling
Cache hit: £ ({})
Currency Columns (1.5ms) SHOW FIELDS FROM `currencies`
=> #<Currency id: 1, name: "GBP", symbol: "\xC2\xA3", rate_from_gbp: 1.0>
Can you send me a simple Ruby script which reproduces the problem?
gist it or email me at mperham AT gmail if you want it kept private.
Here you go:
When I said simple, I meant something that does not require the full Rails stack. I can't put that in a simple unit test for memcache-client.
For instance, this works on HEAD for me:
def test_rails
sym = "£"
puts sym.encoding
m = MemCache.new 'localhost'
p(m.fetch(sym) do
123
end)
end
Try this:
Bump. Anything on this?
Encoding needs to be the first line in the source file.
I can reproduce it, but no guarantees when I'll be able to get it fixed. Maybe tomorrow night.
I'd be happy to fix it if I knew more about the interaction between the client and memcache server. However, I'm not that versed. Could you point me in the right direction, and then maybe I can fix it and you can pull?