Antlr4: How to exit a grammar rule? - antlr4

Question

So I"m experimenting with Antlr v4, and I'm poking it with some unusual grammar to get a sense of how it works. Here's my current test case:
I'd like a grammar that consists of the letters A, B, C, D in that order. The letters may be repeated. I also group the A's and B's together, and the C's and D's also, to make the grammar more interesting. So strings like these are acceptable grammars:
AAA
ABCD
ACCCDD
But it's not going well. I think what is happening is that Antlr needs a better exit rule for my grammar. It doesn't seem to recognize that after collecting the A's and B's, that the presence of a C means to go to the next rule. Actually it's sort of working, but I get error messages, and the resulting parse tree seems to have null elements in it, like it inserted an extra element where it issued the error message.
Here's an example error message:
line 1:2 extraneous input 'C' expecting {'B', 'A'}
which happens for the input 'ABCD'. So something weird is going on when Antlr sees the C there. Here's the output of the parse tree:
'ABCD': (prog (aOrB (a A) (aOrB (b B) aOrB)) (cOrD (c C) (cOrD (d D) cOrD)) <EOF>)
which you can see has an empty aOrB element there at the end of the first set of elements.
Any idea what is going on? What is Antlr "thinking" here when it issues the error and adds the empty element? And how might I fix this?
OK, here are the gory details.
My grammar:
grammar Abcd;
prog : aOrB cOrD EOF;
aOrB : ( a | b ) aOrB ;
a : 'A'+ ;
b : 'B'+ ;
cOrD : ( c | d ) cOrD ;
c : 'C'+ ;
d : 'D'+ ;
My test program in Java:
package antlrtests;
import antlrtests.grammars.*;
import org.antlr.v4.runtime.*;
import org.antlr.v4.runtime.tree.*;
class AbcdTest {
private final String[] testVectors = {
"A", "AABB", "B", "ABCD", "C", "D", };
public void runTests() {
for( String test : testVectors )
simpleTest( test );
}
private void simpleTest( String test ) {
ANTLRInputStream ains = new ANTLRInputStream( test );
AbcdLexer wpl = new AbcdLexer( ains );
CommonTokenStream tokens = new CommonTokenStream( wpl );
AbcdParser wikiParser = new AbcdParser( tokens );
ParseTree parseTree = wikiParser.prog();
System.out.println( "'" + test + "': " + parseTree.toStringTree(
wikiParser ) );
}
}
And the output of my test program. Note the error message are jumbled up with the regular output because they are printed by Antlr on standard error.
run:
line 1:1 no viable alternative at input '<EOF>'
'A': (prog (aOrB (a A) aOrB) cOrD <EOF>)
line 1:4 no viable alternative at input '<EOF>'
'AABB': (prog (aOrB (a A A) (aOrB (b B B) aOrB)) cOrD <EOF>)
'B': (prog (aOrB (b B) aOrB) cOrD <EOF>)
line 1:1 no viable alternative at input '<EOF>'
line 1:2 extraneous input 'C' expecting {'B', 'A'}
line 1:4 no viable alternative at input '<EOF>'
'ABCD': (prog (aOrB (a A) (aOrB (b B) aOrB)) (cOrD (c C) (cOrD (d D) cOrD)) <EOF>)
line 1:0 no viable alternative at input 'C'
line 1:1 no viable alternative at input '<EOF>'
line 1:0 no viable alternative at input 'D'
'C': (prog aOrB (cOrD (c C) cOrD) <EOF>)
line 1:1 no viable alternative at input '<EOF>'
'D': (prog aOrB (cOrD (d D) cOrD) <EOF>)
BUILD SUCCESSFUL (total time: 0 seconds)
Any help is much appreciated.

Answer 1

Is this not what you're after?
prog : 'A'* 'B'* 'C'* 'D'* EOF;
The following rule of your grammar matches an infinitely long series of A and B tokens because the tail recursive aOrB reference is not optional. Your grammar will either throw a StackOverflowException if the input starts sufficiently many A and/or B characters, or encounter a syntax error if it does not.
aOrB : ( a | b ) aOrB ;
If you want to maintain the groupings, you could use this grammar instead. I only made changes to the aOrB and cOrD rules. Since the a rule matches a sequence of A tokens, the aOrB rule uses a? instead of a* (only one instance of a could ever appear, and the entire series of A tokens would be its children).
grammar Abcd;
prog : aOrB cOrD EOF;
aOrB : a? b?;
a : 'A'+ ;
b : 'B'+ ;
cOrD : c? d?;
c : 'C'+ ;
d : 'D'+ ;
Here is another grammar that matches the same language (but produces a different parse tree) showing other options for the *, +, and ? quantifiers. I wouldn't recommend using this grammar, but you should look over it very carefully to understand what each choice is doing and understand why it matches exactly the same input as the grammar I gave above.
grammar Abcd;
prog : aOrB cOrD? EOF;
aOrB : a* b;
a : 'A' ;
b : 'B'* ;
cOrD : (c d* | d+);
c : 'C'+ ;
d : 'D' ;

Answer 2

Did you realize that your aOrB rule doesn't enforce any ordering of a's and b's? Likewise your cOrD rule.