antlr4ide/antlr4ide

Parse Tree Viewer can't find lexer tokens in non-combined grammar

david-hall opened this issue · 0 comments

Using the two Antlr4 Grammars shown below, the Syntax Diagram views for both of them work fine, but the Parse Tree viewer throws this exception in the Console window as soon as I type anything for it to parse. In addition, the Eclipse status bar shows a moving progress bar next to the words "building parse tree scheduler", but that task continues to run after throwing the exception until I close the Parse Tree window.

Here's the exception:

error(160): CalcParser.g4:4:15: cannot find tokens file .\CalcLexer.tokens
Exception in thread "Thread-20" java.lang.NullPointerException
	at org.antlr.v4.runtime.atn.ATNSerializer.serialize(ATNSerializer.java:89)
	at org.antlr.v4.runtime.atn.ATNSerializer.getSerialized(ATNSerializer.java:547)
	at org.antlr.v4.runtime.atn.ATNSerializer.getSerializedAsChars(ATNSerializer.java:551)
	at org.antlr.v4.tool.Grammar.createParserInterpreter(Grammar.java:1138)
	at com.github.jknack.antlr4ide.runtime.ParseTreeCommand.run(ParseTreeCommand.java:80)
	at com.github.jknack.antlr4ide.runtime.Antlr4Server.run(Antlr4Server.java:66)

If I combine them both into a single file, then it all works as expected. The real problem is that I am working with another more substantial grammar where I also have the lexer and parse grammar in separate files and I really can't combine them just to use the Parse Tree view.

Both of these grammar files are in the same folder, but the generated code (java, interp and tokens files) is in a different folder from the g4 files. The project builds correctly within Eclipse, NetBeans or from the command line, and the Syntax Diagram finds everything too. It's just the Parse Tree view in Eclipse Oxygen that's having trouble finding the tokens file.

Any suggestions?

Here are the separate grammar files that are failing:
CalcLexer.g4:

lexer grammar CalcLexer;

NUMBER
  : DIGIT+ '.' DIGIT*
  | '.' DIGIT+
  | DIGIT+
  ;

fragment
DIGIT: ('0'..'9');

WS: [ \t\r\n]+ -> skip;
MULTIPLY: '*';
DIVIDE: '/';
ADD: '+';
SUBTRACT: '-';
LEFT: '(';
RIGHT: ')';

CalcParser.g4:

parser grammar CalcParser; 
 
options {  
    tokenVocab=CalcLexer; 
}  

calculator
  : expression
  ;

expression
  : expression operator=('*'|'/') expression  # MultiplyDivide
  | expression operator=('+'|'-') expression  # AddSubtract
  | '-' expression                            # Negate
  | NUMBER                                    # Number
  | '(' expression ')'                        # Parenthesis
  ;

Here's the combined grammar which works fine:

grammar Calculator;

calculator
  : expression
  ;

expression
  : expression operator=('*'|'/') expression  # MultiplyDivide
  | expression operator=('+'|'-') expression  # AddSubtract
  | '-' expression                            # Negate
  | NUMBER                                    # Number
  | '(' expression ')'                        # Parenthesis
  ;

NUMBER
  : DIGIT+ '.' DIGIT*
  | '.' DIGIT+
  | DIGIT+
  ;

fragment
DIGIT: ('0'..'9');

WS: [ \t\r\n]+ -> skip;
MULTIPLY: '*';
DIVIDE: '/';
ADD: '+';
SUBTRACT: '-';
LEFT: '(';
RIGHT: ')';