bug: Conventional gitmoji commits to file fails to generate changelog due to encoding error

[markus-work]

Description of the bug

Outputing changelog to file with conventional commits and a non-ascii character in the commit message will result in the error:

git-changelog: 'charmap' codec can't encode character '\U0001f527' in position 212: character maps to <undefined>

To Reproduce

1. Clone this repository
2. Create virtual environment
3. Install the requirements from the requirements.txt
4. Run the command git-changelog --config-file config/git-changelog-md.toml
5. Observe the error 'charmap' codec can't encode character
6. Run the command git-changelog --config-file config/git-changelog-no-output.toml
7. Observe no error occurs.
8. Install the fixed version of git-changelog pip install -r requirements-fixed.txt
9. Run the command git-changelog --config-file config/git-changelog-md.toml
10. Observe no error occurs and changelog.md is created.

Full traceback

Full traceback

$ git-changelog --config-file config/git-changelog-md.toml 
git-changelog: 'charmap' codec can't encode character '\u2795' in position 212: character maps to <undefined>

Expected behavior

Changelog generated as normal.

Environment information

$ git-changelog --debug-info  # | xclip -selection clipboard
- __System__: Windows-10-10.0.19045-SP0
- __Python__: cpython 3.10.11 (C:\repos\mre-git-changelog\.venv\Scripts\python.exe)
- __Environment variables__:
- __Installed packages__:
  - `git-changelog` v2.5.2

Additional context

I have also reproduced it once in the actions, but I can't get it reliably to be reproduced. Locally on my machine it happens every time.

I have explicitly set the encoding here, and that works every time: https://github.com/markus-work/git-changelog / #84

[pawamoy]

Hey @markus-work, thanks for the report!

On mobile right now, sorry for the short answer: does it work if you set the following environment variable?

PYTHONUTF8=1

Or this less modern one:

PYTHONIOENCODING=UTF-8

[markus-work]

Thanks for the quick reply!

Yes, it does work for PYTHONUTF8=1. Not for PYTHONIOENCODING=UTF-8.

[pawamoy]

Sorry I meant PYTHONIOENCODING=UTF8 (mobile auto-correction...). But in any case, since PYTHONUTF8=1 makes it work, I'll consider it to be the solution. I very rarely encounter Python tools providing an encoding CLI flag or API option. Therefore I'll close your PR, I hope you don't mind 🙂 Feel free to comment further of course!