Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add functions that suggests where to continue after invalid UTF-8/UTF-16 sequences #2

Open
tormol opened this issue Jul 25, 2016 · 2 comments

Comments

@tormol
Copy link
Owner

tormol commented Jul 25, 2016

...To make it easy to continue if you're ignoring errors or inserting replacement characters.

This requires an error overhaul as many errors already include a position, and having it in two places would be dumb.
On the other hand, if we return (*Error,usize) the errors Display impl cannot use the usize.

Idea:struct SomeError<E>{err:E, len:usize}

@tormol tormol changed the title from_utf{8,16}_slice() should return length on error from_utf8_slice() should return length on error Jun 2, 2017
@tormol
Copy link
Owner Author

tormol commented Jun 2, 2017

For UTF-16 the next unit might always be (the start of) a valid codepoint.
... unless the error is that the second unit is a leading surrogate, and and the unit after it also is a leading surrogate; then ignoring the second unit would be more correct, but that seems unlikely.

@tormol tormol changed the title from_utf8_slice() should return length on error Add functions that suggests where to continue after invalid UTF-8/UTF-8 sequences Oct 15, 2018
@tormol
Copy link
Owner Author

tormol commented Oct 15, 2018

I've decided to only advance one u8/u16 per returned error for predictability, but there could be helper functions that do this.

@tormol tormol changed the title Add functions that suggests where to continue after invalid UTF-8/UTF-8 sequences Add functions that suggests where to continue after invalid UTF-8/UTF-16 sequences Mar 22, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant