pytest-flakes doesn't respect PEP 263 encoding markers

[The-Compiler]

I noticed pytest-flakes doesn't respect PEP 263 encoding markers.

This means with a non-UTF8 aware locale (e.g. LC_ALL=C) or with an OS which doesn't use UTF-8 by default (e.g. latin1 on Windows), files containing UTF-8 won't be checkable:

$ LC_ALL=C ./venv/bin/py.test --flakes snowman.py -rs
[...]
platform linux -- Python 3.4.3 -- py-1.4.27 -- pytest-2.7.1
[...]
venv/lib/python3.4/site-packages/pytest_flakes.py:75: in runtest
    found_errors, out = check_file(self.fspath, self.flakesignore)
venv/lib/python3.4/site-packages/pytest_flakes.py:124: in check_file
    codeString = path.read()
venv/lib/python3.4/site-packages/py/_path/common.py:133: in read
    return f.read()
venv/lib/python3.4/encodings/ascii.py:26: in decode
    return codecs.ascii_decode(input, self.errors)[0]
E   UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 29: ordinal not in range(128)

snowman.py:

# encoding=utf-8

snowman = "☃"

A normal flake8 snowman.py works fine.

This breaks pytest's tests with tox >= 2.0.0 as it isolates environment variables by default.

A possible fix would be to use tokenize.open:

--- pytest_flakes.py.ori    2015-05-22 07:46:50.005433885 +0200
+++ pytest_flakes.py    2015-05-22 07:47:35.018767377 +0200
@@ -4,6 +4,7 @@
 import py
 import pytest
 import sys
+import tokenize


 def assignment_monkeypatched_init(self, name, source):
@@ -121,7 +122,8 @@


 def check_file(path, flakesignore):
-    codeString = path.read()
+    with tokenize.open(str(path)) as f:
+        codeString = f.read()
     filename = py.builtin._totext(path)
     errors = []
     try:

That seems to fix it for Python 3 - but I haven't tested Python 2 yet. Maybe it's enough to just use normal reading there, or maybe tokenize.open has to be backported, like coverage.py did.

[fschulze]

Can you check the PR I made?

[fschulze]

Fix released with 1.0.0.