Tuple struct index inference

[robojumper]

The first commit adds a helper struct ast::FieldKind to facilitate inference.

The second commit adds a slightly modified test from #1109 while mentioning that there is a problem with how we're handling tuple indexing / floats.

cc #1109

[flodiebold]

LGTM, I don't know how to handle the error recovery though... I'd suggest adding a parser test for the x.0. error recovery though, even if we don't handle it very well currently.

[robojumper]

I'd suggest adding a parser test for the x.0. error recovery though

Done. It had to be a failure test because the validation logic I added rejects attempts to index tuples with anything other than decimal integers without underscores or suffixes, as per the Rust reference.

[matklad]

I think we should avoid intorducing a TupleIndex node, I think it won't be generally useful.

Instead, I suggest hand-writing an

fn index_token(&self) -> Option<SyntaxToken>

method manually, which looks for token among children_with_tokens

We might also add an

enum FieldKind {
    Name(NameRef),
    Index(SyntaxToken)
}

and corresponding accessor for convenience.

[matklad]

The second commit verifies that this doesn't fix autocompletion because we still parse a float literal

Fascinating! So what happens is that, when we insert a fake identifier, we parse self.0.fakeIdent correctly. However, when we than try to lookup the receiver in the original file, we parse self.0. as self, . and 0., which is not unreasonable.

I guess an immediate thing we can do is to rewrite the completion test such that there's at least one char after dot, likeself.0.a. Fixing general tuple fields handling seems more important than fixing this particular error recovery quirk.

As for fixing the quirk.... I honestly don't immediately see a clean solution here. We can tweak the lexer to not lex 0. as a floating point literal, if the preceding token is ., but looking at the previous token changes the mode the lexer operates in significantly. I would game for maybe remapping the token from FP literal to an int and a dot at the parser level? This would still be a horrible hack, but at least it would be at the level of abstraction where looking at the context is ok?

[robojumper]

@matklad I have incorporated the feedback from your comments -- ast::FieldKind is now there to facilitate inference and validation. I have also modified the test to verify that tuple struct dot completion generally works, and left a bunch of FIXMEs in the code that need to be touched if one were to hack around the lexer behavior.

[matklad]

bors r+

[bors]

Build succeeded

• continuous-integration/travis-ci/push

[matklad]

Ah, remapping tokens on the parser level won't be trivial because parser intentionally doesn't know about the token's text...

[flodiebold]

Hm, couldn't we just handle this during lowering to HIR? I think it should just work if we removed the. there...

[matklad]

We'll need to tweak both HIR and completion_ctx. SPecifically, this line in context is currently failing: https://github.com/rust-analyzer/rust-analyzer/blob/990e74ba7c77485f914434ac6f09a40d1364634d/crates/ra_ide_api/src/completion/completion_context.rs#L214

Thinking about it more, I don't like the idea of hacking either lexer (b/c that adds context to lexer) or parser (b/c it adds back knowledge about token texts to parser), so I am indeed inclined towards patching this (in a pretty horrible way, by looking at the sibling token of field expressions) when converting to HIR / completing.