Yelp/yelp-python

Search API/Unicode Error

Closed this issue · 6 comments

jrs477 commented

Greetings,

I am able to construct an API client OK. However, when I search the Yelp API with
params = { 'term': 'food', 'lang': 'fr' }
client.search('San Francisco', **params)

I get the following error:
TypeError: character mapping must return integer, None or unicode

I am trying to call the API from France. Would my location have anything to do with this error?

Thanks for any tips. I am new to Python and a bit overwhelmed by documentation concerning unicode errors.

mittonk commented

Hm, that query works for me:

params = {'term': 'food', 'lang': 'fr'}                                         
results = client.search('San Francisco', **params)                                
print results                                                                   
print results.businesses[0].name                                                

<yelp.obj.search_response.SearchResponse object at 0x7f994130f210>
HRD
  1. What's the full stack trace? It'd be very helpful to see exactly which line is triggering the error.
  2. What version of Python are you using?
  3. What sort of OS (Windows, Linux, Mac, etc)?
jrs477 commented

Here is the first part of my script:

import csv
import json

from yelp.client import Client
from yelp.oauth1_authenticator import Oauth1Authenticator

with open('keys.json') as creds_file:
    creds = json.load(creds_file)
    auth = Oauth1Authenticator(**creds)

client = Client(auth)

params = {
    'term': 'food',
    'lang': 'fr'
}

client.search('San Francisco', **params)

Here is the full traceback when I run that section alone:
Traceback (most recent call last): File "clientlib_sample.py", line 23, in <module> client.search('San Francisco', **params) File "/Library/Python/2.7/site-packages/yelp/endpoint/search.py", line 44, in search self.client._make_request(SEARCH_PATH, url_params) File "/Library/Python/2.7/site-packages/yelp/client.py", line 47, in _make_request signed_url = self.authenticator.sign_request(url, url_params) File "/Library/Python/2.7/site-packages/yelp/oauth1_authenticator.py", line 34, in sign_request self.token File "/Library/Python/2.7/site-packages/oauth2/__init__.py", line 335, in sign_request self['oauth_signature'] = signature_method.sign(self, consumer, token) File "/Library/Python/2.7/site-packages/oauth2/__init__.py", line 654, in sign hashed = hmac.new(key, raw, hashlib.sha1) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hmac.py", line 136, in new return HMAC(key, msg, digestmod) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hmac.py", line 75, in __init__ self.outer.update(key.translate(trans_5C)) TypeError: character mapping must return integer, None or unicode
When I comment out client.search('San Francisco', **params), the script executes without an error.

I am running the script in OSX Yosemite.

My version of python is:
2.7.10 (default, Jul 14 2015, 19:46:27)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)]

mittonk commented

Three-backticks for a more readable stacktrace:

Traceback (most recent call last):
  File "clientlib_sample.py", line 23, in <module>
    client.search('San Francisco', **params)
  File "/Library/Python/2.7/site-packages/yelp/endpoint/search.py", line 44, in search
    self.client._make_request(SEARCH_PATH, url_params)
  File "/Library/Python/2.7/site-packages/yelp/client.py", line 47, in _make_request
    signed_url = self.authenticator.sign_request(url, url_params)
  File "/Library/Python/2.7/site-packages/yelp/oauth1_authenticator.py", line 34, in sign_request
    self.token
  File "/Library/Python/2.7/site-packages/oauth2/__init__.py", line 335, in sign_request
    self['oauth_signature'] = signature_method.sign(self, consumer, token)
  File "/Library/Python/2.7/site-packages/oauth2/__init__.py", line 654, in sign
    hashed = hmac.new(key, raw, hashlib.sha1)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hmac.py", line 136, in new
    return HMAC(key, msg, digestmod)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hmac.py", line 75, in __init__
    self.outer.update(key.translate(trans_5C))
TypeError: character mapping must return integer, None or unicode

Looks a lot like

I suspect reading the OAuth creds from a JSON file is giving Unicode rather than bytestrings; HMAC is choking on that. Poking.

mittonk commented

Got it, I can repro issue with oauth2==1.0.0 but not oauth2==1.9.0.post1.

Definitely need to have creds as Unicode (not bytestring) to trigger the error.

@jrs477 : As a workaround, try pip install oauth2==1.9.0.post1, see if that helps you --- I'll track down more specifics.

jrs477 commented

@mittonk Thank you. My calls to the API work. What's special about oauth2==1.9.0.post1?

mittonk commented

@jrs477 : Looks like the oauth2 library got more robust to Unicode between 1.2.0 and 1.5.150. Looks like any version after 1.5.150 works with your code, doesn't necessarily have to be 1.9.0.post1.

I checked a few versions of oauth2 (hooray for binary search), to see how they handle Unicode auth credentials; 1.2.0 crashes while 1.5.150 works as expected. Sometime between those two versions, oauth2 got more careful about handling Unicode creds.

Since supplying creds as a bytestring works as expected on 1.0.0 and later, I don't think we want to restrict the version in yelp-python's setup.py... but if lots of people hit this it might be worth it.

To sum up:
Workaround 1: Use oauth2 1.5.150 or newer. This is the easiest approach.
Workaround 2: Supply your OAuth creds as bytestrings rather than Unicode (see json or general Python docs for more info.)