Add trim/0, ltrim/0 and rtrim/0 that trims leading and trailing whitespace #3056
Conversation
src/jv_unicode.c
Outdated
| @@ -118,3 +118,16 @@ int jvp_utf8_encode(int codepoint, char* out) { | |||
| assert(out - start == jvp_utf8_encode_length(codepoint)); | |||
| return out - start; | |||
| } | |||
|
|
|||
| // space codepoints for unicode basic latin and latin-1 supplement | |||
There was a problem hiding this comment.
This is same as what golang strings.TrimSpace/unicode.IsSpace considers whitespace https://pkg.go.dev/unicode#IsSpace
There was a problem hiding this comment.
Note that Go's strings.TrimSpace considers White_Space property as well.
docs/content/manual/manual.yml
Outdated
| `"\f"`, | ||
| `"\u000b"` (vertical tab), | ||
| `"\u0085"` (next line) and | ||
| `"\u00a0"` (no-break space). These are the whitespace characters in |
There was a problem hiding this comment.
Is this exhaustive for Unicode, or just Latin scripts in Unicode? If the latter, why not be more exhaustive?
There was a problem hiding this comment.
Is only whitespace from basic block and latin-1, so not exhaustive. Reason is mostly to match what other implementations do. There are quite a lot of other whitespace characters in other blocks, wikipedia has a good list https://en.wikipedia.org/wiki/Whitespace_character.
Reading about PCRE's \s-class it seem to match something similar.
There was a problem hiding this comment.
Rust trim/is_whitespace uses characters with White_Space property https://www.unicode.org/Public/UCD/latest/ucd/PropList.txt maybe more resonable?
There was a problem hiding this comment.
Maybe we should document that the list is not stable, that we may add to it.
There was a problem hiding this comment.
Rust
trim/is_whitespaceuses characters withWhite_Spaceproperty https://www.unicode.org/Public/UCD/latest/ucd/PropList.txt maybe more resonable?
I do like that better, yeah, especially in light of @itchyny's comment.
There was a problem hiding this comment.
Wait for utf8proc to be included for upcase/downcase?
There was a problem hiding this comment.
Update to White_Space but i guess we can wait. Also update docs and tests.
|
I found a list of languages on this function. I have no objection to name this trim not strip, because we already have ltrimstr. Does anyone want ltrim and rtrim as well? |
Personally mostly have needed |
wader
force-pushed
the
trim
branch
2 times, most recently
from
48a7b8c
to
913457b
Compare
wader
changed the title
|
Added |
|
@emanuele6 thanks for review! maybe wait for one more apporval? thinking adding new functions might be good idea with more consensus About waiting for utf8proc from #2547: i think we can merge this separately and fix that later |
Yeap, had a quick look at it a few week ago and it seemed like it would be easy, something like |
Trims leading and trailing whitespace. Was added to jq in jqlang/jq#3056
No description provided.