List of small errors to fix and possible improvements to implement

[Delgan]

I prefer to gather everything here rather than opening a ticket for each of them, as some bugs are really minimal. Some points are just ideas I have had.

• Nested issues are not displayed (see)
• Surround repr(value) with try / except just in case something goes wrong (see traceback source code)
• Print Traceback (most recent call last): right before formatting the issue, so user have a clue about what is going wrong if formatting hangs (see #37)
• \n inside a string should be escaped, no newline should be displayed (see)
• SyntaxError are wrongly truncated (see and see #35)
• Multilines expressions like a = 1 + [2, \n 3] are not enhanced
• Encoding issues:
  • "天" / 1 is wrongly displayed as "天" / 11 because of utf-8 string slicing (see)
  • Do not force char decoding, stay close to the default behavior displaying "\u5929" instead of "天" if LANG = en_US.ascii (see)
  • Keep the quotation marks and u string prefixes used in the source code, do not replace them with ' marks (see)
  • Encoding long strings may raise an error (see #34)
• Possible readability improvement:
  • Syntax highlighting could (optionally) include operators (*, +, -, /, ...), functions, methods
  • int and string values could be highlighted with two different colors
  • Traceback frames could be optionally spaced between each others (see)
  • Is it possible for shadowed built-in (len = lambda x: 5) to not be higlighted as a built-in?
  • Location (File "test.py", line 13 in func) could be colored (see #38)
  • Other messages (like Exception: XXX, Traceback (most recent call last):, ...) could be optionally colored too
  • Traceback relevant to the user could be bold / not relevant could be dimmed (see #38)
  • Advanced user could have more control over formatting to suit its need with colors which works best on its terminal, or by customizing style (see #39). For example: import better_exceptions; better_exceptions.THEME['location_message'] = 'File "{filepath} +{lineno}", in {source}'. Maybe use ansimarkup to easily colorize / strip (if SUPPORTS_COLORS is False) messages.
  • c = self.a + self.b: is it possible to give information about a and b? It seems very complicated. What about dct['key'], lst[3], obj.val, a.b.c, x.get('key', default), etc.

[Delgan]

Hi @Qix-. I remember reading on Reddit that you wanted to avoid any dependency on another library. Is it still a goal?

I ask that because I see two libraries that might be useful.

Pygments for syntax highlighting of source lines.
Using it would simplify the code by removing all parts related to the AST. This fixes the bugs related to UTF-8 slicing and quotation marks, this avoids some encoding issues, this fixes multilines expressions not colored, this gives more control over which tokens can be colored.

ansimarkup for other colored messages.
The idea is to ease the coloring of messages, with for example 'File <green>"{filepath}"</green>, line <yellow>{lineno}</yellow>, in <magenta>{source}</magenta>'. Tags are more eloquent than raw ansi code. The other interest is to be able to easily remove the colors if the terminal does not support them, using ansimarkup.strip(string) before .format().

I would understand if you prefer not to add dependency. I would then try to code a solution otherwise. But I prefer to ask what you like best before working on it.

[Qix-]

Nah both of them are completely bloated and huge, and ansimarkup is absolutely the wrong way to do ANSI escapes IMO.

So yes, no-dependencies is absolutely a goal here :)

[Delgan]

@Qix- Fine, I understand. 👍

I do not want to abuse your time but I'm a little curious to know your point of view: why do you think that ansimarkup is not a good way to manage ANSI codes?

[Qix-]

@Delgan It's just convoluted and bloated IMO. Having dealt with ANSI escapes a lot over at Chalk, the last thing I'd want to introduce into my text rendering was HTML or XML-like tags.

ANSI escapes aren't always closures, either - there are better ways to transition to a new type than to saturate the output with a bunch of close/open codes. For example, you can switch directly from one color to another color without resetting the color in-between. Most 'wrappers' for ANSI escapes generally don't do that and emit a few more bytes just to cancel the color directly before changing it to another one.

[Delgan]

Thank you for the explanation. ;)

I thought it was the easiest way to deal with ANSI codes, but I now realize that it's not very clean to mix the markup directly with the string.

[Qix-]

Yeah absolutely :) I think the current highlighter implementation is more than enough. I battled with whether or not I wanted to add it at all but got drunk and did it anyway.

[Delgan]

Haha. I find the syntax highlighting rather welcome and enjoyable. 👍

I thought it could easily be extended, but it appears that this is not the case.
Indeed, it is better to leave it as such, it's not worth the trouble.

[Qix-]

Yeah, looking at this a year and a half later, unfortunately the highlighter right now is as feature-full as possible without adding a custom Python parser (the built-in parsing system isn't good enough to get definitive results beyond what is already there).

[Qix-]

@Delgan how many of these are still issues in only python 3? I know that a lot of the encoding issues go away in python 3.

[Qix-]

Also, to expand a bit on the goal of better-exceptions - it is meant to be a debugging utility, not anything that is polished for production (please, please don't use this module in production - I cannot be held liable for data leaks in your logs).

Therefore, there's no reason for it to bring in all of the extra bloat just for a few more pieces of highlighted text.

In fact, the missing highlighted parts can be fixed - it's just a matter of rewriting the existing syntax highlighting code.

Feel free to open a ticket for that specifically.

[Delgan]

@Qix- I ticked the fixed issues. Actually, there isn't many "Python 3 only" bugs.

Some of the encoding issues I listed are in fact no so much related to encoding. I discovered this was because of the way the highlighter formatted strings.

Looking at this one year and a half later, I couldn't agree more that wanting to add dependencies just for fixing that was foolish. 🙂

I managed to fixes several issues by using the built-in tokenize module instead of ast. I think it's easier to work with a flatten list of tokens than an syntax tree, but the main advantage is that it keep the displayed source code as is (while ast doesn't give access to it), this is why it resolves some problems with string formatting.

I planned to open a pull request but needed first to fix my problems with Python 2.7: Delgan@674b529

All of the other issues listed here are actually fixed in one of my branches or commits, I just have to clean them up a little before opening pull requests.

[Qix-]

You can submit the PR and have it work only with python 3 since we're deprecating 2. It's not a critical feature enough to bend over backwards to make it work on 2.