tasos-py/Search-Engines-Scraper

Please support python 3.8

reisenmachtfreude opened this issue · 2 comments

With python 3.8.5 on ubuntu this scaper cannot be used:
Error:

Traceback (most recent call last):
  File "search.py", line 4, in <module>
    results = engine.search("Something")
  File "/usr/local/lib/python3.8/dist-packages/search_engines-0.5-py3.8.egg/search_engines/engine.py", line 161, in search
    response = self._get_page(request['url'], request['data'])
  File "/usr/local/lib/python3.8/dist-packages/search_engines-0.5-py3.8.egg/search_engines/engine.py", line 66, in _get_page
    return self._http_client.get(page)
  File "/usr/local/lib/python3.8/dist-packages/search_engines-0.5-py3.8.egg/search_engines/http_client.py", line 21, in get
    page = self._quote(page)
  File "/usr/local/lib/python3.8/dist-packages/search_engines-0.5-py3.8.egg/search_engines/http_client.py", line 41, in _quote
    if utl.decode_bytes(utl.unquote_url(url)) == utl.decode_bytes(url):
  File "/usr/local/lib/python3.8/dist-packages/search_engines-0.5-py3.8.egg/search_engines/utils.py", line 15, in unquote_url
    return decode_bytes(requests.utils.unquote(url))
AttributeError: module 'requests.utils' has no attribute 'unquote'

Content of file search.py

from search_engines import Bing

engine = Bing()
results = engine.search("Something")
links = results.links()
print(links)

This doesn't look like a Python version issue. If you notice your traceback, you'll see that an AttributeError is raised, bcause it can't import unquote from requests.utils. Which is strange, because in https://github.com/psf/requests/blob/master/requests/utils.py#L30 they import unquote, so we should be able to import unquote from requests.utils. What version or requests are you using?

Thanks for your answer. After upgrading from Ubuntu 20.04 to 21.04 with python 3.9.5 the issue is solved for me.