Visualize strings / standalone visualizer

[zspitz]

This is the shared workflow for:

• a Visual Studio debugging visualizer attached to the string type
• a standalone application that shows the parse tree for user-input source code, and a selected lexer and parser

User	Code
Choose lexer and parser assembly and class
Optionally, choose rule
Either enter source code, or load from a file (within the debugging visualizer, this would be set by calling code)
Click Parse
	Instantiate lexer, common token source, parser
	Create instance of VisualizerData class from returned parser rule context
	Show visualized

@bclothier @retailcoder Does the above work for visualizing strings?

[bclothier]

I think it would. I am assuming that for the first 2 steps, that can be saved to a settings, so that we don't have to do it every time we visualize a string.

Also, because I'm unfamiliar with visualizers in general, how does that work when a string type has multiple visualizer? I think by default it has a zoom window. I think that should stay as the default since it's only a subset of all possible string literals/variables that needs to be visualized in this manner.

I like the idea of being able to load a file in addition to pasting arbitrary text into a standalone application for a quick preview of the tree. I have a hunch that if I'm writing this from scratch and I'm doing lot of tweaks to my parser, I might need to be able to easily reload the parser/lexer after making changes to the grammar. Therefore, I see two possibilities:

1. instead of selecting the lexer/parser, allow us to select the .g4 grammar files, and use Sam's fork to autogenerate the lexer/parser and use that instead. The downside is that it's slower because you're now building a mini-project as part of parsing but that also makes for an independent assembly to verify the grammar directly from the .g4 file, which would able us to experiment with changes and assessing the impact of a change without going the trouble in our own project. Consider that Rubberduck currently has 18 + 3 projects that needs to be built and the Rubberduck.Parsing is fairly low in the build chain. That adds up to much time just to see the change.

2. If you don't want to deal with building the lexer/parser directly from the grammar files, maybe make it easy to reload the classes if you detect that the DLL files has been changed? In our case, that means we have to get into habit of only building the Rubberduck.Parsing project to effect this for our experimentation.

[zspitz]

I am assuming that for the first 2 steps, that can be saved to a settings, so that we don't have to do it every time we visualize a string.

Yes, absolutely.

when a string type has multiple visualizers

Visual Studio has this little magnifying glass + dropdown, when hovering over an expression, or from the Watch and Locals panes. When you click on the magnifying glass, the previously used visualizer is shown, When you click on the dropdown, you can choose any of the installed visualizers for that particular type.

RE reloading parser -- If the lexer or parser is coming from a DLL, this isn't an issue -- the DLL should be loaded only in order to parse and then unloaded. If the DLL is rebuilt in the meantime, the application would use the new DLL.

RE lexing/parsing directly from .g4 file -- There's an additional tradeoff, because sometimes the lexer or parser class has some additional functionality which isn't available from the .g4 file alone.

[bclothier]

I think that if the .g4 file has the option section, you can issue a warning that the representation won't be accurate and simply ignore the option section altogether. At least the user is informed but still can do ad-hoc parsing for quick & dirty tweaking & testing. That workflow was used in Eclipse and was helpful for refining the challenging parts of the grammar.

[zspitz]

I think that if the .g4 file has the option section, you can issue a warning that the representation won't be accurate and simply ignore the option section altogether.

I was thinking more of VBAParser being a partial class, and thus can have additional, non-autogenerated members.

The only way I can see to lex/parse directly from the g4 file, is to code-gen C# files from the g4 file (either using Java, or Sam Harwell's .NET code-gen fork), and then to runtime-compile those files using Roslyn. In that case, wouldn't the option section be a part of the code-gen?

In the far future I might consider writing an interpreter class for ANTLR grammars, that would create an actual instance of a parser class entirely at runtime, without needing any code-gen. (Probably using expression trees.)

@bclothier If you have any other ides on how to go about this, I'd be very interested.

[bclothier]

You are correct about codegen creating the classes from the ‘option‘. Given that the classes are always ‘partial ‘, I think it is given that such visualization will be incomplete. As an option, you could enable the user to select an assembly instead of the codegen to get the full fidelity for the visualization.