Unicode decoding error under Windows

[florian-huber]

First off, many thanks for the really nice package!

I started working with it for a collection of Python teaching material in markdown format where I would like to test (some of) the code blocks.
When adding tests for Windows I ran into the following error:

UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 76: character maps to <undefined>

I later realized it comes from a markdown file that had some non-intended characters (from copy-pasting code):

s = “my text”

In a fork of pytest-codeblocks I now added macos and Windows to the CI and a test running on an example markdown file with the problematic characters.
The error can be "fixed" by adding a 'replace' to the open() function. Could be there's better ways to handle this, but it at least worked for me. --> florian-huber#5

Let me know if that would be interesting for a pull request. Thanks!

[nschloe]

Well, it's illegal python so it's a good thing it fails, right?

s = “my text”
  File "<ipython-input-1-d2d48acdde47>", line 1
    s = “my text”
        ^
SyntaxError: invalid character '“' (U+201C)

[florian-huber]

Good point :)

I should have made a better case I guess. The actual example where I ran into this issue was a failing test. I guess it should happen for any also if you had the character as a comment (say s = "string" # don't use other characters such as: “my text”...)
If it would fail during the execution of the python code that would of course be totally fine (desired even!). But now it failed at the open(...) stage.

But maybe that makes it more of a corner case...

[nschloe]

Fix released.

[florian-huber]

Great, thanks a lot @nschloe !