UnicodeEncodeError on valid RSS feed
Closed this issue · 2 comments
samuelclay commented
Nothing special going on, here's the repro:
>>> url = "https://news.google.com/news/rss/local/section/geo/Valbonne,%20France/Valbonne,%20Provence-Alpes-Côte%20d%27
Azur,%20France?ned=fr&hl=fr&gl=FR"
>>> import feedparser
>>> feedparser.parse(url)
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/Users/sclay/.virtualenvs/newsblur3/lib/python3.9/site-packages/feedparser/api.py", line 214, in parse
data = _open_resource(url_file_stream_or_string, etag, modified, agent, referrer, handlers, request_headers, result)
File "/Users/sclay/.virtualenvs/newsblur3/lib/python3.9/site-packages/feedparser/api.py", line 114, in _open_resource
return http.get(url_file_stream_or_string, etag, modified, agent, referrer, handlers, request_headers, result)
File "/Users/sclay/.virtualenvs/newsblur3/lib/python3.9/site-packages/feedparser/http.py", line 158, in get
f = opener.open(request)
File "/usr/local/Cellar/python@3.9/3.9.0_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 517, in open
response = self._open(req, data)
File "/usr/local/Cellar/python@3.9/3.9.0_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 534, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "/usr/local/Cellar/python@3.9/3.9.0_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 494, in _call_chain
result = func(*args)
File "/usr/local/Cellar/python@3.9/3.9.0_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 1385, in https_open
return self.do_open(http.client.HTTPSConnection, req,
File "/usr/local/Cellar/python@3.9/3.9.0_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 1342, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "/usr/local/Cellar/python@3.9/3.9.0_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1255, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/Cellar/python@3.9/3.9.0_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1266, in _send_request
self.putrequest(method, url, **skips)
File "/usr/local/Cellar/python@3.9/3.9.0_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1104, in putrequest
self._output(self._encode_request(request))
File "/usr/local/Cellar/python@3.9/3.9.0_4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1184, in _encode_request
return request.encode('ascii')
UnicodeEncodeError: 'ascii' codec can't encode character '\xf4' in position 79: ordinal not in range(128)
>>> feedparser.__version__
'6.0.2'
And here's the valid RSS: https://validator.w3.org/feed/check.cgi?url=https%3A%2F%2Fnews.google.com%2Frss%2Ftopics%2FCAAqIggKIhxDQkFTRHdvSkwyMHZNRGhvTlRGb0VnSmxiaWdBUAE%3Fhl%3Dfr%26gl%3DFR%26ceid%3DFR%3Afr
kurtmckee commented
@samuelclay, I am going to rip out all HTTP client code in feedparser and use the requests module for very basic functionality in the future.
However, feedparser shouldn't crash so I'll investigate and release a hotfix soon.
kurtmckee commented
This is fixed in feedparser 6.0.4. Thanks for reporting this, Samuel!