Skip to content

Commit 38ff99a

Browse files
committed
Modify algorithm to allow repetitions after NTs.
1 parent 2356418 commit 38ff99a

File tree

1 file changed

+28
-23
lines changed

1 file changed

+28
-23
lines changed

text/0550-macro-future-proofing.md

Lines changed: 28 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -74,31 +74,36 @@ The algorithm for recognizing valid matchers `M` follows. Note that a matcher
7474
is merely a token tree. A "simple NT" is an NT without repetitions. That is,
7575
`$foo:ty` is a simple NT but `$($foo:ty)+` is not. `FOLLOW(NT)` is the set of
7676
allowed tokens for the given NT's fragment specifier, and is defined below.
77-
`F` is used for representing the separator in complex NTs. In `$($foo:ty),+`,
78-
`F` would be `,`, and for `$($foo:ty)+`, `F` would be `EOF`.
7977

80-
*input*: a token tree `M` representing a matcher and a token `F`
81-
82-
*output*: whether M is valid
83-
84-
For each token `T` in `M`:
85-
86-
1. If `T` is not an NT, continue.
87-
2. If `T` is a simple NT, look ahead to the next token `T'` in `M`. If
88-
`T'` is `EOF` or a close delimiter of a token tree, replace `T'` with
89-
`F`. If `T'` is in the set `FOLLOW(NT)`, `T'` is EOF, or `T'` is any close
90-
delimiter, continue. Otherwise, reject.
91-
3. Else, `T` is a complex NT.
92-
1. If `T` has the form `$(...)+` or `$(...)*`, run the algorithm on the
93-
contents with `F` set to the token following `T`. If it accepts,
94-
continue, else, reject.
95-
2. If `T` has the form `$(...)U+` or `$(...)U*` for some token `U`, run
96-
the algorithm on the contents with `F` set to `U`. If it accepts,
97-
check that the last token in the sequence can be followed by `F`. If
98-
so, accept. Otherwise, reject.
78+
`CHECK(M, F):` *input*: a sequence of tokens `M` and a set of tokens `F`; *output*: whether M is valid.
79+
1. If `M` is empty, accept.
80+
2. Set `t = HEAD(M)`
81+
3. If `t` is not an NT, skip to 7.
82+
4. Find `S`, the set of possible successors of `t`:
83+
1. If `TAIL(M)` is empty, set `S = F`,
84+
2. Else, `S = LEADERS(TAIL(M))`.
85+
5. If `t` is a simple NT, check that `S` is a subset of `FOLLOW(t)`.
86+
If so, skip to 7, else, reject.
87+
6. Else, `t` is a complex NT.
88+
1. If `t` has the form `$(Q)+` or `$(Q)*`, run `CHECK(Q, S)`.
89+
If it accepts, skip to 7, else, reject.
90+
2. If `t` has the form `$(Q)u+` or `$(Q)u*` for some token `u`,
91+
run `CHECK(Q, S + {u})` If it accepts, skip to 7, else, reject.
92+
7. Set `M = TAIL(M)`, goto 1.
93+
94+
`LEADERS(S):` Returns the set of all possible tokens that may begin input sequence matched by `S`.
95+
1. If `S` is empty, return `{}`.
96+
2. Set `t = HEAD(S)`
97+
3. If `t` is not a complex NT, return `{t}`.
98+
4. If `t` is a complex NT:
99+
1. If `t` has the form `$(Q)+` or `$(Q)u+`, return `LEADERS(Q)`.
100+
2. If `t` has the form `$(Q)*` or `$(Q)u*`, return `LEADERS(Q) + LEADERS(TAIL(S))`.
101+
102+
`HEAD(M)` Returns the first token in sequence `M`.
103+
`TAIL(M)` Returns sequence `M` with the first token removed.
99104

100105
This algorithm should be run on every matcher in every `macro_rules`
101-
invocation, with `F` as `EOF`. If it rejects a matcher, an error should be
106+
invocation, with `F` as `{}` (empty set). If it rejects a matcher, an error should be
102107
emitted and compilation should not complete.
103108

104109
The current legal fragment specifiers are: `item`, `block`, `stmt`, `pat`,
@@ -115,7 +120,7 @@ The current legal fragment specifiers are: `item`, `block`, `stmt`, `pat`,
115120
- `FOLLOW(item)` = any token
116121
- `FOLLOW(meta)` = any token
117122

118-
(Note that close delimiters are valid following any NT.)
123+
(Note that open and close delimiters are valid following any NT.)
119124

120125
# Drawbacks
121126

0 commit comments

Comments
 (0)