Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Do NFKC normalization in lexer #802

Closed
steveklabnik opened this issue Feb 2, 2015 · 2 comments
Closed

Do NFKC normalization in lexer #802

steveklabnik opened this issue Feb 2, 2015 · 2 comments
Labels
T-compiler Relevant to the compiler team, which will review and decide on the RFC. T-lang Relevant to the language team, which will review and decide on the RFC.

Comments

@steveklabnik
Copy link
Member

Issue by catamorphism
Thursday Apr 19, 2012 at 22:13 GMT

For earlier discussion, see rust-lang/rust#2253

This issue was labelled with: A-frontend, A-unicode, I-crash, I-enhancement, P-low in the Rust repository

next_token_inner in the lexer has a comment saying to do NKFC normalization. I have no idea what that is, but I guess we should do it.

reference: NFKC is one of four unicode Normalization Forms.

reference: UAX-31 supplies guidelines for use of normalization with identifiers.
@steveklabnik steveklabnik mentioned this issue Feb 2, 2015
Closed
@m13253
Copy link

m13253 commented Feb 4, 2015

I support NFC instead of NFKC.

Because I don't think we should normalize "⒛" into "20.".

@pnkfelix pnkfelix mentioned this issue Nov 1, 2015
Closed
@petrochenkov petrochenkov added T-lang Relevant to the language team, which will review and decide on the RFC. T-compiler Relevant to the compiler team, which will review and decide on the RFC. labels Jan 28, 2018
@Centril
Copy link
Contributor

Centril commented Oct 7, 2018

Closing in favor of #2457.

@Centril Centril closed this as completed Oct 7, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
T-compiler Relevant to the compiler team, which will review and decide on the RFC. T-lang Relevant to the language team, which will review and decide on the RFC.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@steveklabnik @Centril @m13253 @petrochenkov