Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Take encoding into account in string functions. #4111

Merged
merged 4 commits into from
Aug 25, 2024
Merged

Take encoding into account in string functions. #4111

merged 4 commits into from
Aug 25, 2024

Conversation

toots
Copy link
Member

@toots toots commented Aug 25, 2024

  • Add optional encoding for string.length and string.sub, default to "utf8"
  • Add string.chars with encoding.
  • Fix default string escaping to properly fallback to "ascii" when utf8 escaping failed.

A global settings.string.default_encoding option is added to make it possible to revert to the previous behavior and assume "ascii" encoding for the impacted functions (string.length and string.sub).

Fixes: #4109

  `"utf8"`
* Add `string.chars` with encoding.
* Fix default string escaping to properly fallback to `"ascii"` when
  utf8 escaping failed.

Fixes: #4109
@toots toots enabled auto-merge August 25, 2024 20:27
@toots toots added this pull request to the merge queue Aug 25, 2024
@toots toots removed this pull request from the merge queue due to a manual request Aug 25, 2024
@toots toots enabled auto-merge August 25, 2024 21:48
@toots toots added this pull request to the merge queue Aug 25, 2024
Merged via the queue into main with commit 3b4b9a7 Aug 25, 2024
25 checks passed
@toots toots deleted the utf-utils branch August 25, 2024 22:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

String functions UTF-8 issue
1 participant