-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[JavaScript-runtime] Export ATN transitions with types #4805
base: dev
Are you sure you want to change the base?
Conversation
Signed-off-by: Matthias Harzer <[email protected]>
Signed-off-by: Matthias Harzer <[email protected]>
Signed-off-by: Matthias Harzer <[email protected]>
Signed-off-by: Matthias Harzer <[email protected]>
@MatthiasHarzer thanks for this, but this is internal stuff. Exporting it would create an obligation to maintain it. What problem are you trying to solve ? |
Hi there @ericvergnaud, As I mentioned in my issue #4810, I'm trying to build a suggestion system to recommend tokens, that could follow a partial input. I really like the idea of leveraging the existing ANTLR ATN for this, as ANTLR is already a direct dependency, and I want to keep external dependencies to a minimum. Using the ATN also ensures that any changes made to the grammar ( Projects like antlr4-c3 use the ATN to predict following tokens and rules from an ANTLR-grammar and a partial input. However, I hesitate to use such tool as it depends on an alternative ANTLR-generator antlr4ng. My goal is to build on the official ANTLR target and runtime as they appear to be supported the best. Additionaly, JavaScript isn't the only target in my custom language and I strongly prefer using the official ANTLR generator for all targets. Exporting and typing these classes would not impact the rest of the runtime, but it would enable features like the one I’m trying to build. In my opinion, the benefits of making them available outweigh the maintenance overhead. I hope this clarifies my reasoning, and I welcome any further questions or feedback. |
Another application of the ATN data structures is to enumerate all the ambiguous parse trees at decision states. The code I wrote to do this, in C# and part of a generated driver app for testing grammars in Trash, is used instead of the erroneous one currently in the CSharp runtime. I could not do this without access to the ATN data structures. To do this tree enumeration, you also need the DecisionInfo data structure, aka "profiling", to know which decision states are ambiguous. Unfortunately, that's not implemented in the JS/TS runtime. If profiling and ambiguous parse tree enumeration are not part of the runtime (and they should not), then give us the tools so we can implement them. Once enumerated, Trash has about a half-dozen different ways to display the parse trees, which are much faster and more intuitive than generating .dot files and trying to find the decision state by eyeball. Ambiguity in grammars-v4 is pervasive: out of 363 grammars, 215 are ambiguous. |
@MatthiasHarzer Some time ago I started looking at implementing a code-completion core. The code is here https://github.com/ericvergnaud/antlr4/tree/suggestion-helper. The intent is precisely to provide a built-in tool for suggestions without exhibiting all internals of ANTLR (which create stickiness). It's still early days but maybe it's worth glancing at it ? |
The current JS runtime does not export ATN transitions, which makes it hard to walk the ATN from outside.
This PR adds types for the ATN transitions (
src/antlr4/transition
) as well as exporting them so they can be imported from outside.closes #4810