ANTLR 4: Dynamic Token - antlr4

Question

Given:
grammar Hbs;
var: START_DELIM ID END_DELIM;
START_DELIM: '{{';
END_DELIM: '}}';
I'd like to know how to change START_DELIM and END_DELIM at runtime to be for example <% and %>.
Does anyone know how to do this in ANTLR 4?
Thanks.

Answer 1

There's a way, but you'll need to tie your grammar to a target language (as of now, the only target is Java).
Here's a quick demo (I included some comments to clarify things):
grammar T;
#lexer::members {
// Some default values
private String start = "<<";
private String end = ">>";
public TLexer(CharStream input, String start, String end) {
this(input);
this.start = start;
this.end = end;
}
boolean tryToken(String text) {
// See if `text` is ahead in the CharStream.
for(int i = 0; i < text.length(); i++) {
if(_input.LA(i + 1) != text.charAt(i)) {
// Nope, we didn't find `text`.
return false;
}
}
// Since we found the text, increase the CharStream's index.
_input.seek(_input.index() + text.length() - 1);
return true;
}
}
parse
: START ID END
;
START
: {tryToken(start)}? .
// The `.` is needed because a lexer rule must match at least 1 char.
;
END
: {tryToken(end)}? .
;
ID
: [a-zA-Z]+
;
SPACE
: [ \t\r\n] -> skip
;
The { ... }? is a semantic predicate. See: https://github.com/antlr/antlr4/blob/master/doc/predicates.md
Here's a small test class:
import org.antlr.v4.runtime.*;
import org.antlr.v4.runtime.tree.*;
public class Main {
private static void test(TLexer lexer) throws Exception {
TParser parser = new TParser(new CommonTokenStream(lexer));
ParseTree tree = parser.parse();
System.out.println(tree.toStringTree(parser));
}
public static void main(String[] args) throws Exception {
// Test with the default START and END.
test(new TLexer(new ANTLRInputStream("<< foo >>")));
// Test with a custom START and END.
test(new TLexer(new ANTLRInputStream("<? foo ?>"), "<?", "?>"));
}
}
Run the demo as follows:
*nix
java -jar antlr-4.0-complete.jar T.g4
javac -cp .:antlr-4.0-complete.jar *.java
java -cp .:antlr-4.0-complete.jar Main
Windows
java -jar antlr-4.0-complete.jar T.g4
javac -cp .;antlr-4.0-complete.jar *.java
java -cp .;antlr-4.0-complete.jar Main
And you'll see the following being printed to the console:
(parse << foo >>)
(parse <? foo ?>)