Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple emoji encoding issues on Windows #67

Open
tmr232 opened this issue Sep 13, 2022 · 3 comments
Open

Multiple emoji encoding issues on Windows #67

tmr232 opened this issue Sep 13, 2022 · 3 comments

Comments

@tmr232
Copy link

tmr232 commented Sep 13, 2022

When trying to use rich to print files with emoji on Windows, there are some encoding issues.

Below are 2 cases I encountered.

Garbled text instead of emoji

When running rich broken-emoji.md (broken-emoji.md - a text file with nothing but the 😊 emoji in it) on Windows (in Windows Terminal), I get the following:

😊

If I run Get-Content broken-emoji.md or run rich inside WSL, I get the emoji printed as expected.

Rich fails to print entirely

When running rich cannot-print.md (cannot-print.md - only contains the 🤝 emoji) on Windows, I get:

unable to read .\cannot-print.md: 'charmap' codec can't decode byte 0x9d in position 3: character maps to <undefined>

Running it in WSL or using Get-Content cannot-print.md in the same terminal window gives me the emoji as expected.

Expected Results

As this works in the same terminal both with Powershell's Get-Content, and when using WSL to run rich-cli, I'd expect it to work in Windows as well.

Environment

OS: Windows 10 (build 19044.1889)
Terminal: Windows Terminal (version 1.14.2281.0) running PowerShell
Rich CLI: 1.8.0
Python: 3.10.1

@tmr232
Copy link
Author

tmr232 commented Sep 13, 2022

Update: This seems to be caused by the current codepage not being 65001 (UTF-8).
Setting $env:PYTHONUTF8=1 solves this.

I'm leaving this issue open for 2 reasons:

  1. Maybe there's a way around it, to make it work by default
  2. I assume more people encounter the same issue, and having it documented / printing a suggestion when the error occurs could be useful.

@Pebaz
Copy link

Pebaz commented Sep 29, 2022

Ran into this also, the $env:PYTHONUTF8=1 trick worked for me, although it would be awesome if it could "just work" ™️. It might not be useful, but it looks like you can invoke python with python -X utf8 ... and it does the same trick @tmr232 showed above without modifying the environment.

@tmr232
Copy link
Author

tmr232 commented Oct 5, 2022

Just verified now - setting the system locale to support UTF8 fixes it as well (as expected), but I'd still prefer rich to "just work" when using Windows Terminal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants