Skip to content

Improve parser (recovery) for expressions #11188

Closed
@Veykril

Description

@Veykril

The general approach here is to look at our syntax tree (show syntax tree), at the InteliiJ syntax tree (PSI viewer plugin), at the Kotlin parser, and make the recovery work.

I don't think we should special-case the arg list parsing though, Kotlin doesn't seem to do that:

https://github.com/JetBrains/kotlin/blob/dc568426bd4b73a377187c32991ffd98e9ec1106/compiler/psi/src/org/jetbrains/kotlin/parsing/KotlinExpressionParsing.java#L1825

From what I've looked at, the problem is our expression parsing. Here are some problemeatic cases:

  • fn main() { call(x::) } -- here, the ) get's eaten by PATH_EXPR, this shouldn't be happening. Culprit is somewhere around here. It semes that for expression paths we shoudl be using a different recovery set
  • fn main() { f(x+) } is a similar case, binary expressions eats )
  • fn main() { f(x:) } this one is fun! I think this one works in IntelliJ Rust, becaues it supprts type ascription, and we don't. I think what I'd expect to see here is that the whole x: gets parsed as a qualified path.

On trick we should steal from kotlin for arg list itself is this condition:
https://github.com/JetBrains/kotlin/blob/dc568426bd4b73a377187c32991ffd98e9ec1106/compiler/psi/src/org/jetbrains/kotlin/parsing/KotlinExpressionParsing.java#L1839-L1842

Originally posted by @matklad in #10195 (comment)
Looked some more into this, not super trivial to fix. The problem is, to make a resonable recovery in (expr,*), we need first to make sure that expr parsing itself is reasonable. Today it is not: in (1+), expression will eat ). Just fixing that by adding ) to

https://github.com/rust-analyzer/rust-analyzer/blob/a6e6f9c58b951459a73497618fc523d96de24fba/crates/parser/src/grammar/expressions/atom.rs#L62

doesn't work -- it actually breaks our stmt* parsing in blocks, which assumes that statement always consumes at least one token. It seems like we want to remove recovery from here:

https://github.com/rust-analyzer/rust-analyzer/blob/a6e6f9c58b951459a73497618fc523d96de24fba/crates/parser/src/grammar/expressions/atom.rs#L150-L152

and instead add it to the toplevel, to whatever code actually parses expressions. Will try to investigate further.

Originally posted by @matklad in #10195 (comment)

cc #10410 #10173

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-parserparser issuesC-enhancementCategory: enhancementE-unknownIt's unclear if the issue is E-hard or E-easy without digging in

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions