kurtmckee/feedparser

seems to return empty feed in case of network errors

Opened this issue · 0 comments

I use this indirectly via Feediverse (an RSS to Fediverse poster), and I just had that post an entire feed’s worth of posts again.

What I think happened here is that the network was down (the provider is under DDoS or something) and…

        try:
            return opener.open(request)
        finally:
            opener.close() # JohnD

    # try to open with native open function (if url_file_stream_or_string is a filename)
    try:
        return open(url_file_stream_or_string, 'rb')
    except (IOError, UnicodeEncodeError, TypeError):
[…]
        pass

    # treat url_file_stream_or_string as string
    if isinstance(url_file_stream_or_string, str):
        return _StringIO(url_file_stream_or_string.encode('utf-8'))
    return _StringIO(url_file_stream_or_string)

… this sequence of ignoring the exceptions made it parse the URL as string and thus return an empty feed.

Then, the next time the feed was readable again, it got all posts again, and since it didn’t know about them (it only remembers the last ones for this specific feed as that feed doesn’t use dates), it posted them all again.

Please just throw an exception when a network feed is not reachable. Feediverse will then exit with an uncaught exception, which will have cron send me an eMail, and that’s just perfect and the way I expected things to work.

(If you have a fix, backporting it against 5.2.1 for us Debian users would be great!)