@@ -74,31 +74,36 @@ The algorithm for recognizing valid matchers `M` follows. Note that a matcher
74
74
is merely a token tree. A "simple NT" is an NT without repetitions. That is,
75
75
` $foo:ty ` is a simple NT but ` $($foo:ty)+ ` is not. ` FOLLOW(NT) ` is the set of
76
76
allowed tokens for the given NT's fragment specifier, and is defined below.
77
- ` F ` is used for representing the separator in complex NTs. In ` $($foo:ty),+ ` ,
78
- ` F ` would be ` , ` , and for ` $($foo:ty)+ ` , ` F ` would be ` EOF ` .
79
77
80
- * input* : a token tree ` M ` representing a matcher and a token ` F `
81
-
82
- * output* : whether M is valid
83
-
84
- For each token ` T ` in ` M ` :
85
-
86
- 1 . If ` T ` is not an NT, continue.
87
- 2 . If ` T ` is a simple NT, look ahead to the next token ` T' ` in ` M ` . If
88
- ` T' ` is ` EOF ` or a close delimiter of a token tree, replace ` T' ` with
89
- ` F ` . If ` T' ` is in the set ` FOLLOW(NT) ` , ` T' ` is EOF, or ` T' ` is any close
90
- delimiter, continue. Otherwise, reject.
91
- 3 . Else, ` T ` is a complex NT.
92
- 1 . If ` T ` has the form ` $(...)+ ` or ` $(...)* ` , run the algorithm on the
93
- contents with ` F ` set to the token following ` T ` . If it accepts,
94
- continue, else, reject.
95
- 2 . If ` T ` has the form ` $(...)U+ ` or ` $(...)U* ` for some token ` U ` , run
96
- the algorithm on the contents with ` F ` set to ` U ` . If it accepts,
97
- check that the last token in the sequence can be followed by ` F ` . If
98
- so, accept. Otherwise, reject.
78
+ ` CHECK(M, F): ` * input* : a sequence of tokens ` M ` and a set of tokens ` F ` ; * output* : whether M is valid.
79
+ 1 . If ` M ` is empty, accept.
80
+ 2 . Set ` t = HEAD(M) `
81
+ 3 . If ` t ` is not an NT, skip to 7.
82
+ 4 . Find ` S ` , the set of possible successors of ` t ` :
83
+ 1 . If ` TAIL(M) ` is empty, set ` S = F ` ,
84
+ 2 . Else, ` S = LEADERS(TAIL(M)) ` .
85
+ 5 . If ` t ` is a simple NT, check that ` S ` is a subset of ` FOLLOW(t) ` .
86
+ If so, skip to 7, else, reject.
87
+ 6 . Else, ` t ` is a complex NT.
88
+ 1 . If ` t ` has the form ` $(Q)+ ` or ` $(Q)* ` , run ` CHECK(Q, S) ` .
89
+ If it accepts, skip to 7, else, reject.
90
+ 2 . If ` t ` has the form ` $(Q)u+ ` or ` $(Q)u* ` for some token ` u ` ,
91
+ run ` CHECK(Q, S + {u}) ` If it accepts, skip to 7, else, reject.
92
+ 7 . Set ` M = TAIL(M) ` , goto 1.
93
+
94
+ ` LEADERS(S): ` Returns the set of all possible tokens that may begin input sequence matched by ` S ` .
95
+ 1 . If ` S ` is empty, return ` {} ` .
96
+ 2 . Set ` t = HEAD(S) `
97
+ 3 . If ` t ` is not a complex NT, return ` {t} ` .
98
+ 4 . If ` t ` is a complex NT:
99
+ 1 . If ` t ` has the form ` $(Q)+ ` or ` $(Q)u+ ` , return ` LEADERS(Q) ` .
100
+ 2 . If ` t ` has the form ` $(Q)* ` or ` $(Q)u* ` , return ` LEADERS(Q) + LEADERS(TAIL(S)) ` .
101
+
102
+ ` HEAD(M) ` Returns the first token in sequence ` M ` .
103
+ ` TAIL(M) ` Returns sequence ` M ` with the first token removed.
99
104
100
105
This algorithm should be run on every matcher in every ` macro_rules `
101
- invocation, with ` F ` as ` EOF ` . If it rejects a matcher, an error should be
106
+ invocation, with ` F ` as ` {} ` (empty set) . If it rejects a matcher, an error should be
102
107
emitted and compilation should not complete.
103
108
104
109
The current legal fragment specifiers are: ` item ` , ` block ` , ` stmt ` , ` pat ` ,
@@ -115,7 +120,7 @@ The current legal fragment specifiers are: `item`, `block`, `stmt`, `pat`,
115
120
- ` FOLLOW(item) ` = any token
116
121
- ` FOLLOW(meta) ` = any token
117
122
118
- (Note that close delimiters are valid following any NT.)
123
+ (Note that open and close delimiters are valid following any NT.)
119
124
120
125
# Drawbacks
121
126
0 commit comments