Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use UTF-8 for Linux #12

Draft
wants to merge 7 commits into
base: master
Choose a base branch
from
Draft

Use UTF-8 for Linux #12

wants to merge 7 commits into from

Conversation

zeule
Copy link
Contributor

@zeule zeule commented Aug 28, 2024

The wchar_t in Linux is very wide, 32 bits, and thus almost unused,
which, in turn, results in poor support in tooling. And that's to be
expected when UTF-8 is the default. At the same time, Iconv, which is
part of glibc, is universally available. Thus we can use it to convert
UCS-2 into UTF-8 in Linux and always use UTF-8 in the library API.

So that IDEs can list them.
Simplify script, fix install() for the bundled onigiruma, fix using
pkg-config in the find module.
This allows to use find_package() with the build tree.
The wchar_t in Linux is very wide, 32 bits, and thus almost unused,
which, in turn, results in poor support in tooling. And that's to be
expected when UTF-8 is the default. At the same time, Iconv, which is
part of glibc, is universally available. Thus we can use it to convert
UCS-2 into UTF-8 in Linux and always use UTF-8 in the library API.
@zeule
Copy link
Contributor Author

zeule commented Sep 3, 2024

@hzdbyte, which other project do I have to adapt to merge in this change? Only the bundled GUI, or qgen as well, for example?

@hzdbyte
Copy link
Contributor

hzdbyte commented Sep 4, 2024

To be honest I don't really like using iconv as dependency.
The library doesn't strictly depend on wchar_t since it has it's own string processing implementation, thus it can use any type of strings. I'd still prefer to keep the internal string representation based on fixed length type (it can be "short"/2bytes for example). We can implement UTF8 support in the special version of bindings to be used by clients/players with UTF8 strings (that'd be next to the "default" one).

@zeule
Copy link
Contributor Author

zeule commented Sep 5, 2024

But Iconv is a part of glibc, thus it brings no new dependency package-wise. Utf8 is the standard for Linux locales. Can that change your attitude?

@hzdbyte
Copy link
Contributor

hzdbyte commented Sep 5, 2024

Wouldn't it be better to create utf8 binding that'd convert strings back and forth for utf8 players?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants