Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Unicode Input Issue and OSError in Test Script #119

Merged
merged 10 commits into from
Aug 11, 2024

Conversation

sakkyoi
Copy link
Contributor

@sakkyoi sakkyoi commented Aug 10, 2024

No description provided.

@sakkyoi sakkyoi changed the title Fix Unicode Input and OSError Issues in Test Script Fix Unicode Input Issue and OSError in Test Script Aug 10, 2024
Copy link
Collaborator

@Cube707 Cube707 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

those are my imidieate thought on the changes.

Also, how this this new behavior compare to Linux? can all wide-charcters allready be read on linux systems or must changes be also implemented there?

readchar/_win_read.py Outdated Show resolved Hide resolved
readchar/_win_read.py Outdated Show resolved Hide resolved
tests/windows/conftest.py Outdated Show resolved Hide resolved
tests/windows/conftest.py Outdated Show resolved Hide resolved
@sakkyoi
Copy link
Contributor Author

sakkyoi commented Aug 11, 2024

I tested it on macOS without any issues. The original problem seems to occur only on Windows.

@Cube707
Copy link
Collaborator

Cube707 commented Aug 11, 2024

So you tested with relevant characters on MacOS?
But not on Windows?

@sakkyoi
Copy link
Contributor Author

sakkyoi commented Aug 11, 2024

I apologize for my incomplete explanation. I initially noticed this issue on Windows, so I fixed it. However, I also noticed that Windows handles this differently compared to Linux and macOS. I tested it on macOS, and it showed that macOS can natively handle Unicode character input perfectly.

@Cube707
Copy link
Collaborator

Cube707 commented Aug 11, 2024

Took me a while to get the Japanese keyboard to work on windows, but here is what I found:

This seems to mostly work as expected. Linux is fine as is, as you have found with Mac:
grafik

windows work for all characters I tested, but breaks on emojis for some reason:
grafik

@sakkyoi
Copy link
Contributor Author

sakkyoi commented Aug 11, 2024

After my investigation, it appears that some emojis are encoded using UTF-8 surrogates, which may appear as two characters in the terminal (you can observe this if you select them with your cursor). This is related to the issue of "surrogates not allowed." Perhaps you could use a try-except block to handle UnicodeEncodeError, as this error is raised by getwchar().

@sakkyoi
Copy link
Contributor Author

sakkyoi commented Aug 11, 2024

I might have found a solution, so please stay tuned.

@Cube707
Copy link
Collaborator

Cube707 commented Aug 11, 2024

After my investigation, it appears that some emojis are encoded using UTF-8 surrogates, which may appear as two characters in the terminal 

Still strang that it works fine on linux

@sakkyoi
Copy link
Contributor Author

sakkyoi commented Aug 11, 2024

Linux works fine without this issue.

@Cube707
Copy link
Collaborator

Cube707 commented Aug 11, 2024

this seems to be a good solution, but I would prefer that code in the readkey() function, as it handles identically to the combined key-presses. Surrogates are also made up of two chars that don't make sense on their own.

I will implement that now

@coveralls
Copy link

coveralls commented Aug 11, 2024

Coverage Status

coverage: 94.904% (+0.07%) from 94.839%
when pulling 050dd5b on sakkyoi:master
into a3e9b0b on magmax:master.

@Cube707 Cube707 merged commit d1d903d into magmax:master Aug 11, 2024
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants