mperham/memcache-client

Ruby 1.9 compatibility - use bytesize instead of size

Closed this issue · 15 comments

memcache-client 1.8.5 running on Ruby 1.9 can't cope with non-ascii utf-8 strings as data ("values"). The memcached protocol requires the number of bytes, but memcache-client passes on number of characters:

it is using value.to_s.size everywhere, with Ruby 1.9 it should be value.to_s.bytesize.

Feel free to fork, fix and send pull request. Otherwise, I'll get to it one of these days.

Seems like this has been fixed :)

Yep, sorry forgot to mention here. Issue should be fixed in HEAD.

Sorry, false positive. Looks like using Ruby 1.9.1 p378 and Rails 2.3.8, I am still getting the Encoding::CompatibilityError exception when attempting to send UTF-8 values to the memcache-client.

>> Rails.cache.fetch(symbol) { first(:conditions => { :symbol => "£" }) }

Currency Load (0.3ms)   SELECT * FROM `currencies` WHERE (`currencies`.`symbol` = '£') LIMIT 1
Encoding::CompatibilityError: incompatible character encodings: UTF-8 and ASCII-8BIT
from /Users/jdelsman/Ultraspeed/core/vendor/plugins/memcache-client/lib/memcache.rb:374:in `block in set'
from /Users/jdelsman/Ultraspeed/core/vendor/plugins/memcache-client/lib/memcache.rb:892:in `with_server'
from /Users/jdelsman/Ultraspeed/core/vendor/plugins/memcache-client/lib/memcache.rb:367:in `set'
from /Users/jdelsman/.rvm/gems/ruby-1.9.1-p378/gems/activesupport-2.3.8/lib/active_support/cache/mem_cache_store.rb:82:in `write'
from /Users/jdelsman/.rvm/gems/ruby-1.9.1-p378/gems/activesupport-2.3.8/lib/active_support/cache/strategy/local_cache.rb:51:in `write'
from /Users/jdelsman/.rvm/gems/ruby-1.9.1-p378/gems/activesupport-2.3.8/lib/active_support/cache.rb:165:in `fetch'

Try putting:

# encoding: utf-8

at the top of the source file which has the non-ascii string.

Already had that. I applied the following money patch, and now I'm getting ASCII back:

http://gnuu.org/2009/11/06/ruby19-rails-mysql-utf8/

ruby-1.9.1-p378 > Currency.sterling
Cache hit: £ ({})
Currency Columns (1.5ms)   SHOW FIELDS FROM `currencies`
 => #<Currency id: 1, name: "GBP", symbol: "\xC2\xA3", rate_from_gbp: 1.0> 

Can you send me a simple Ruby script which reproduces the problem?

gist it or email me at mperham AT gmail if you want it kept private.

When I said simple, I meant something that does not require the full Rails stack. I can't put that in a simple unit test for memcache-client.

For instance, this works on HEAD for me:

def test_rails
  sym = "£"
  puts sym.encoding

  m = MemCache.new 'localhost'
  p(m.fetch(sym) do
    123
  end)
end

Bump. Anything on this?

Encoding needs to be the first line in the source file.

I can reproduce it, but no guarantees when I'll be able to get it fixed. Maybe tomorrow night.

I'd be happy to fix it if I knew more about the interaction between the client and memcache server. However, I'm not that versed. Could you point me in the right direction, and then maybe I can fix it and you can pull?