Display coding problem

[dcl-lily]

Code：

-- coding:utf-8 --

import better_exceptions
import sys
print(sys.stdout.encoding)
better_exceptions.hook()

foo = 52

def shallow(a, b):
deep(a + b)

def deep(val):
global foo
assert val > 10 and foo == 60

bar = foo - 50
shallow(bar, 15)
shallow(bar, 2)

display:
"C:\Program Files\Python36\python.exe" C:/Users/Alex/PycharmProjects/Test/Test02.py
UTF-8
Traceback (most recent call last):
File "C:/Users/Alex/PycharmProjects/Test/Test02.py", line 20, in
shallow(bar, 15)
�� �� 2
�� <function shallow at 0x000001860AAEF730>
File "C:/Users/Alex/PycharmProjects/Test/Test02.py", line 11, in shallow
deep(a + b)
�� �� �� 15
�� �� 2
�� <function deep at 0x000001860B079620>
File "C:/Users/Alex/PycharmProjects/Test/Test02.py", line 16, in deep
assert val > 10 and foo == 60
�� �� 52
�� 17
AssertionError: assert val > 10 and foo == 60

Process finished with exit code 1

system: windows 10
python: 3.6

[Delgan]

Hi @dcl-lily

The errors are written to the stderr, could you please tell us the value of sys.stderr.encoding?

Also, what is the output of the chcp command on your Windows terminal?

[dcl-lily]

hi
sys.stderr.encoding value UTF-8

PS C:\Users\Alex> chcp
活动代码页: 936
PS C:\Users\Alex> python -m better_exceptions
Python 3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:54:40) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
(BetterExceptionsConsole)

a = 1
b = 'fgh'
c = a + b
Traceback (most recent call last):
File "", line 1, in
c = a + b
�� �� 'fgh'
�� 1
TypeError: unsupported operand type(s) for +: 'int' and 'str'
print(sys.stderr.encoding)
Traceback (most recent call last):
File "", line 1, in
print(sys.stderr.encoding)
NameError: name 'sys' is not defined
import sys
print(sys.stderr.encoding)
utf-8
print(sys.stdin.encoding)
utf-8
print(sys.stdout.encoding)
utf-8

think

[Delgan]

@dcl-lily Thanks for the output.

I am afraid I do not know what is causing the problem.
This looks like an issue with the Windows 10 terminal, but I am not able to reproduce it.

Some investigations idea I found on Google:

• Run python test.py > file.txt and check with a text editor that file.txt is correct
• Change the font of your Windows terminal (try them all)
• Execute the command chcp 65001 before your Python script
• Try using the legacy console
• Launch your script with a Python version < 3.6 (PEP 528)

I will see if I can find some time in the next days to dig a bit deeper about this issue.

[dcl-lily]

hi
used python 3.5 display OK, in version 3.6 to Change the system configuration but not be able to display normally

[chen19901225]

I get the same problem
env: windows7
python: python3.7
chcp: "活动代码页: 936"
"sys.stderr.encoding": "utf-8",
"locale.getpreferredencoding() ": "cp936",
So the problem seem this
file "better_exception/encoding.py" line 10
ENCODING = locale.getpreferredencoding(),
So I replace with
ENCODING = sys.getdefaultencoding()
and it works
hope it works for you too

[dcl-lily]

@chen19901225 think，is good

[Qix-]

@Delgan can you investigate whether or not https://github.com/Qix-/better-exceptions/issues/53#issuecomment-387991544 is a proper solution or a hack? We might need to change it.

[Delgan]

@Qix- Sure, I will take a look this week-end. ;)

[Delgan]

Hey @chen19901225 and @dcl-lily

We would like to merge your fix "upstream", so you can enjoy future releases of better_exceptions without having to maintain your own local copy of the modified library.

As I'm still not able to reproduce the issue, would you please help me by answering a few questions about your configuration?

• What font and size are used in the properties of your Windows terminal (make a screen if you wish)?
• Does your fix replace mojibake characters with proper utf8 └ or with ascii -> fallback?
• What is the output of sys.getdefaultencoding()?

Thanks!

[Qix-]

Closing due to insufficient repro information. Feel free to comment back if you want to add something and we can re-open as necessary.

Thanks for following up @Delgan.

[frostming]

This solution disables the debugging feature in IPython, which complains no traceback is found,

[bytemain]

I meet this problem too.
Window 10
System language: Chinese

---

so I decide to try #issuecomment-387991544's solution,

and it works.

• sys.getdefaultencoding(): >>> utf-8
• locale.getpreferredencoding() >>> cp936

[Delgan]

Thanks for the report @lengthmin!

I'm re-opening this issue, then, since it's clear something should be done on our side to fix it. I will try to find some time to investigate this.

Which Python version are you using please?

[bytemain]

Python 3.7.1
I think the code locale.getpreferredencoding() is to use every country's own encoding type, but most of them don’t support every unicode character completely.
so it better to switch to UTF-8

[Delgan]

As we are writing exceptions on sys.stderr, I thought we may use sys.stderr.encoding. Does it equal to utf8 on your system?

[bytemain]

yes

[Delgan]

Great, thanks!

@Qix- Any thoughts on replacing locale.getpreferredencoding() with sys.stderr.encoding? Can you remember why getpreferredencoding() was used in the first place?

sys.getdefaultencoding() could work too, but it's seems to contain the encoding internally used by CPython to store str objects, so this is not much related to terminal output.

[Qix-]

If it works, then let's do it :)

[herrome]

change the code file "ENCODING = locale.getpreferredencoding()" to "ENCODING = utf-8"
it works

[Qix-]

@herrome It works for you, but doesn't for anyone that is using a different encoding. I wish we lived in a world where utf-8 was used everywhere but that just isn't the case yet.