-
Notifications
You must be signed in to change notification settings - Fork 532
Parse optionals and repeats without regexes #1826
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The motivation for this refactoring is to next add repeats with separators. |
747ca2f
to
7e7e973
Compare
Rather than parsing optionals and repeats fully in the recursive descent style, we were using regular expressions to do part of the matching and parsing. That's fine for what it is, but as we think about extending the grammar language surrounding repeats further, it might be more straightforward for this to be parsed in the more usual way. So let's do that. Doing this also results in better and more targeted errors when parsing malformed syntax. We had been supporting a space between an expression and the optional and repeat sigils `?`, `*`, and `+` (but not between an expression and the `{a..b}` ranged repeat syntax). In making this change, we drop this support and adjust the affected productions. We were only using this in a handful of places, and the clarity of the productions seem the same or better by removing these spaces. We verified that, setting aside the removal of these spaces, the rendered output of the Reference is byte identical before and after this change.
7e7e973
to
3570070
Compare
ehuss
approved these changes
May 15, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
Zalathar
added a commit
to Zalathar/rust
that referenced
this pull request
May 20, 2025
Update books ## rust-lang/book 4 commits in d33916341d480caede1d0ae57cbeae23aab23e88..230c68bc1e08f5f3228384a28cc228c81dfbd10d 2025-05-19 14:25:14 UTC to 2025-05-08 21:28:56 UTC - Chapter 6 from tech review (rust-lang/book#4370) - Chapter 5 from tech review (rust-lang/book#4359) - Chapter 4 from tech review (rust-lang/book#4358) - Chapter 3 from tech review (rust-lang/book#4353) ## rust-lang/reference 12 commits in 387392674d74656f7cb437c05a96f0c52ea8e601..acd0231ebc74849f6a8907b5e646ce86721aad76 2025-05-19 15:41:22 UTC to 2025-05-06 21:36:01 UTC - Add doc for avx512 target features (rust-lang/reference#1778) - Parse grammar without regexes (rust-lang/reference#1827) - Parse optionals and repeats without regexes (rust-lang/reference#1826) - Fix grammar for `RangePatternBound` regarding literals (rust-lang/reference#1825) - Fix grammar for `LiteralPattern` regarding `-` (rust-lang/reference#1824) - Doc: Add the LoongArch stabilized target features (rust-lang/reference#1707) - Fix naked em-dash (rust-lang/reference#1820) - Add missing attribute for statement macros (rust-lang/reference#1819) - Make linked rules are clicked, highlight the color (rust-lang/reference#1817) - Use the reference grammar for inline assembly (rust-lang/reference#1807) - Fix typo in introduction (rust-lang/reference#1810) - Add an example admonition (rust-lang/reference#1812) ## rust-lang/rust-by-example 2 commits in 8a8918c698534547fa8a1a693cb3e7277f0bfb2f..c9d151f9147c4808c77f0375ba3fa5d54443cb9e 2025-05-13 17:49:05 UTC to 2025-05-13 17:48:43 UTC - fix(docs): standardize on `no_run` attribute for documentation examples (rust-lang/rust-by-example#1929) - Fix typo in Japanese translation (rust-lang/rust-by-example#1928)
rust-timer
added a commit
to rust-lang-ci/rust
that referenced
this pull request
May 20, 2025
Rollup merge of rust-lang#141259 - rustbot:docs-update, r=ehuss Update books ## rust-lang/book 4 commits in d33916341d480caede1d0ae57cbeae23aab23e88..230c68bc1e08f5f3228384a28cc228c81dfbd10d 2025-05-19 14:25:14 UTC to 2025-05-08 21:28:56 UTC - Chapter 6 from tech review (rust-lang/book#4370) - Chapter 5 from tech review (rust-lang/book#4359) - Chapter 4 from tech review (rust-lang/book#4358) - Chapter 3 from tech review (rust-lang/book#4353) ## rust-lang/reference 12 commits in 387392674d74656f7cb437c05a96f0c52ea8e601..acd0231ebc74849f6a8907b5e646ce86721aad76 2025-05-19 15:41:22 UTC to 2025-05-06 21:36:01 UTC - Add doc for avx512 target features (rust-lang/reference#1778) - Parse grammar without regexes (rust-lang/reference#1827) - Parse optionals and repeats without regexes (rust-lang/reference#1826) - Fix grammar for `RangePatternBound` regarding literals (rust-lang/reference#1825) - Fix grammar for `LiteralPattern` regarding `-` (rust-lang/reference#1824) - Doc: Add the LoongArch stabilized target features (rust-lang/reference#1707) - Fix naked em-dash (rust-lang/reference#1820) - Add missing attribute for statement macros (rust-lang/reference#1819) - Make linked rules are clicked, highlight the color (rust-lang/reference#1817) - Use the reference grammar for inline assembly (rust-lang/reference#1807) - Fix typo in introduction (rust-lang/reference#1810) - Add an example admonition (rust-lang/reference#1812) ## rust-lang/rust-by-example 2 commits in 8a8918c698534547fa8a1a693cb3e7277f0bfb2f..c9d151f9147c4808c77f0375ba3fa5d54443cb9e 2025-05-13 17:49:05 UTC to 2025-05-13 17:48:43 UTC - fix(docs): standardize on `no_run` attribute for documentation examples (rust-lang/rust-by-example#1929) - Fix typo in Japanese translation (rust-lang/rust-by-example#1928)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Rather than parsing optionals and repeats fully in the recursive descent style, we were using regular expressions to do part of the matching and parsing. That's fine for what it is, but as we think about extending the grammar language surrounding repeats further, it might be more straightforward for this to be parsed in the more usual way. So let's do that. Doing this also results in better and more targeted errors when parsing malformed syntax.
We had been supporting a space between an expression and the optional and repeat sigils
?
,*
, and+
(but not between an expression and the{a..b}
ranged repeat syntax). In making this change, we drop this support and adjust the affected productions. We were only using this in a handful of places, and the clarity of the productions seem the same or better by removing these spaces.We verified that, setting aside the removal of these spaces, the rendered output of the Reference is byte identical before and after this change.
cc @ehuss