I need to implement generics with Antlr4. In order to do this, I need to be able to take a class and, as it is used, dynamically generate code for it like a macro, tokenize that code, generate a tree, and then add that new tree to my original parse tree.
I saw these two classes
http://www.antlr.org/api/JavaTool/org/antlr/v4/runtime/RuleContext.html
http://www.antlr.org/api/JavaTool/org/antlr/v4/runtime/ParserRuleContext.html
However, I'm not sure what they actually do, nor am I sure how to use the constructor.
ParserRuleContext(ParserRuleContext parent, int invokingStateNumber)
RuleContext(RuleContext parent, int invokingState)
Specifically, are these the classes that will represent the new tree, and what should I pass into invokingState/invokingStateNumber?
Apparently it is as easy as this
When walking the first tree, it correctly displays information from both files.
public class Program
{
private static ParserRuleContext getTree(String file) throws Exception
{
InputStream input = new FileInputStream(file);
Reader reader = new InputStreamReader(input);
ANTLRInputStream inputStream = new ANTLRInputStream(reader);
JavaLexer lexer = new JavaLexer(inputStream);
CommonTokenStream tokens = new CommonTokenStream(lexer);
JavaParser parser = new JavaParser(tokens);
ParserRuleContext tree = parser.compilationUnit(); // parse
return tree;
}
public static void main(String args[]) throws Exception
{
ParserRuleContext tree1 = getTree("E:\\Users\\nessy\\IdeaProjects\\AntlrTest\\src\\in1.java");
ParserRuleContext tree2 = getTree("E:\\Users\\nessy\\IdeaProjects\\AntlrTest\\src\\in2.java");
tree1.addChild(tree2);
ParseTreeWalker walker = new ParseTreeWalker(); // create standard walker
JavaWalker extractor = new JavaWalker();
walker.walk(extractor, tree1); // initiate walk of tree with listener
}
}
Related
So I have recently migrated to v6 and I will try to simplify my question
I have the following class
#AllArgsConstructor
public class Songs {
String title;
List<String> genres;
}
In my scenario I want to have something like:
Then The results are as follows:
|title |genre |
|happy song |romance, happy|
And the implementation should be something like:
#Then("Then The results are as follows:")
public void theResultsAreAsFollows(Songs song) {
//Some code here
}
I have the default transformer
#DefaultParameterTransformer
#DefaultDataTableEntryTransformer(replaceWithEmptyString = "[blank]")
#DefaultDataTableCellTransformer
public Object transformer(Object fromValue, Type toValueType) {
ObjectMapper objectMapper = new ObjectMapper();
return objectMapper.convertValue(fromValue, objectMapper.constructType(toValueType));
}
My current issue is that I get the following error: Cannot construct instance of java.util.ArrayList (although at least one Creator exists)
How can I tell cucumber to interpret specific cells as lists? but keeping all in the same step not splitting apart? Or better how can I send an object in a steps containing different variable types such as List, HashSet, etc.
If I do a change and replace the list with a String everything is working as expected
#M.P.Korstanje thank you for your idea. If anyone is trying to find a solution for this here is the way I did it as per suggestions received. Inspected to see the type fromValue has and and updated the transform method into something like:
if (fromValue instanceof LinkedHashMap) {
Map<String, Object> map = (LinkedHashMap<String, Object>) fromValue;
Set<String> keys = map.keySet();
for (String key : keys) {
if (key.equals("genres")) {
List<String> genres = Arrays.asList(map.get(key).toString().split(",", -1));
map.put("genres", genres);
}
return objectMapper.convertValue(map, objectMapper.constructType(toValueType));
}
}
It is somehow quite specific but could not find a better solution :)
I am loading a groovy script and I need the following class:
import java.util.function.Function
import java.util.function.Supplier
class MaskingPrintStream extends PrintStream {
private final Function<String,String> subFunction
private final Supplier<String> secretText
public MaskingPrintStream(PrintStream out, Supplier<String> secretText, Function<String,String> subFunction) {
super(out);
this.subFunction = subFunction;
this.secretText = secretText;
}
#Override
public void write(byte b[], int off, int len) {
String out = new String(b,off,len);
String secret = secretText.get();
byte[] dump = out.replace(secret, subFunction.apply(secret)).getBytes();
super.write(dump,0,dump.length);
}
}
right now I have it in a file called MaskingPrintStream.groovy. But, by doing this I effectively can only access this class as an inner class of the class that gets created by default that corresponds to the file name.
What I want to work is code more like this:
def stream = evaluate(new File(ClassLoader.getSystemResource('MaskingPrintStream.groovy').file))
But as you can see, I need to give it some values before it's ready. Perhaps I could load the class into the JVM (not sure how from another groovy script) and then instantiate it the old-fashioned way?
The other problem is: How do I set it up so I don't have this nested class arrangement?
groovy.lang.Script#evaluate(java.lang.String) has a bit different purpose.
For your case you need to use groovy.lang.GroovyClassLoader that is able to parse groovy classes from source, compile and load them. Here's example for your code:
def groovyClassLoader = new GroovyClassLoader(this.class.classLoader) // keep for loading other classes as well
groovyClassLoader.parseClass('MaskingPrintStream.groovy')
def buf = new ByteArrayOutputStream()
def maskingStream = new MaskingPrintStream(new PrintStream(buf), { 'secret' }, { 'XXXXX' })
maskingStream.with {
append 'some text '
append 'secret '
append 'super-duper secret '
append 'other text'
}
maskingStream.close()
println "buf = ${buf}"
And the output it produces in bash shell:
> ./my-script.groovy
buf = some text XXXXX super-duper XXXXX other text
I have an antlr grammar in which embedded actions are used to collect data bottom up and build aggregated data structures. A short version is given below, where the aggregated data structures are only printed (ie no classes are created for them in this short sample code).
grammar Sample;
top returns [ArrayList l]
#init { $l = new ArrayList<String>(); }
: (mid { $l.add($mid.s); } )* ;
mid returns [String s]
: i1=identifier 'hello' i2=identifier
{ $s = $i1.s + " bye " + $i2.s; }
;
identifier returns [String s]
: ID { $s = $ID.getText(); } ;
ID : [a-z]+ ;
WS : [ \t\r\n]+ -> skip ;
Its corresponding Main program is:
public class Main {
public static void main( String[] args) throws Exception
{
SampleLexer lexer = new SampleLexer( new ANTLRFileStream(args[0]));
CommonTokenStream tokens = new CommonTokenStream( lexer );
SampleParser parser = new SampleParser( tokens );
ArrayList<String> top = parser.top().l;
System.out.println(top);
}
}
And a sample test is:
aaa hello bbb
xyz hello pqr
Since one of the objectives of antlr is to keep the grammar file reusable and action-independent, I am trying to delete the actions from this file and move it to a tree walker. I took a first stab at it with the following code:
public class Main {
public static void main( String[] args) throws Exception
{
SampleLexer lexer = new SampleLexer( new ANTLRFileStream(args[0]));
CommonTokenStream tokens = new CommonTokenStream( lexer );
SampleParser parser = new SampleParser( tokens );
ParseTree tree = parser.top();
ParseTreeWalker walker = new ParseTreeWalker();
walker.walk( new Walker(), tree );
}
}
public class Walker extends SampleBaseListener {
public void exitTop(SampleParser.TopContext ctx ) {
System.out.println( "Exit Top : " + ctx.mid() );
}
public String exitMid(SampleParser.MidContext ctx ) {
return ctx.identifier() + " bye "; // ignoring the 2nd instance here
}
public String exitIdentifier(SampleParser.IdentifierContext ctx ) {
return ctx.ID().getText() ;
}
}
But obviously this is wrong, because at the least, the return types of the Walker methods should be void, so they dont have a way to return aggregated values upstream. Secondly, I dont see a way how to access the "i1" and "i2" from the walker code, so I am not able to differentiate between the two instances of "identifier" in that rule.
Any suggestions on how to separate the actions from the grammar for this purpose?
Should I use a visitor instead of a listener here, since the visitor has the capability of returning values? If I use a visitor, how do I solve the problem of differentiating between "i1" and "i2" (as mentioned above)?
Does a visitor perform its action only at the exit of a rule (unlike the listeners, which exist for both entry and exit)? For example, if I have to initialize the list at the entry of rule "top", how can I do it with a visitor, which executes only at the conclusion of a rule? Do I need a enterTop listener for that purpose?
EDIT: After the initial post, I have modified the rule "top" to create and return a list, and pass this list back to the main program for printing. This is to illustrate why I need an initialization mechanism for the code.
Based on what you are trying to do I think you may benefit from using ANTLR's BaseVisitor Class rather than the BaseListener Class.
Assuming your grammar is this (I generalized it and I'll explain the changes below):
grammar Sample;
top : mid* ;
mid : i1=identifier 'hello' i2=identifier ;
identifier : ID ;
ID : [a-z]+ ;
WS : [ \t\r\n]+ -> skip ;
Then your Walker would look like this:
public class Walker extends SampleBaseVisitor<Object> {
public ArrayList<String> visitTop(SampleParser.TopContext ctx) {
ArrayList<String> arrayList = new ArrayList<>();
for (SampleParser.MidContext midCtx : ctx.mid()) {
arrayList.add(visitMid(midCtx));
}
return arrayList;
}
public String visitMid(SampleParser.MidContext ctx) {
return visitIdentifier(ctx.i1) + " bye " + visitIdentifier(ctx.i2);
}
public String visitIdentifier(SampleParser.IdentifierContext ctx) {
return ctx.getText();
}
}
This allows you to visit and get the result of any rule you want.
You are able to access i1 and i2, as you labeled them through the visitor methods. Note that you don't really need the identifier rule since it contains only one token and you can access a token's text directly in the visitMid, but really it's personal preference.
You should also note that SampleBaseVisitor is a generic class, where the generic parameter determines the return type of the visit methods. For your example I set the generic parameter Object, but you could even make your own class which contains the information you want to preserve and use that for your generic parameter.
Here are some more useful methods which BaseVisitor inherits which may help you out.
Lastly, your main method would end up looking something like this:
public static void main( String[] args) throws IOException {
FileInputStream fileInputStream = new FileInputStream(args[0]);
SampleLexer lexer = new SampleLexer(CharStreams.fromStream(fileInputStream));
CommonTokenStream tokens = new CommonTokenStream(lexer);
SampleParser parser = new SampleParser(tokens);
for (String string : new Walker().visitTop(parser.top())) {
System.out.println(string);
}
}
As a side note, the ANTLRFileStream class is deprecated in ANTLR4.
It is recommend to use CharStreams instead.
As Terence Parr points out in the Definitive Reference, one main difference between Visitor and Listener is that the Visitor can return values. And that can be convenient. But Listener has a place too! What I do for listener is exemplified in this answer. Granted, there are simpler ways of parsing a list of numbers, but I made that answer to show a complete and working example of how to aggregate return values from a listener into a public data structure that can be consumed later.
public class ValuesListener : ValuesBaseListener
{
public List<double> doubles = new List<double>(); // <<=== SEE HERE
public override void ExitNumber(ValuesParser.NumberContext context)
{
doubles.Add(Convert.ToDouble(context.GetChild(0).GetText()));
}
}
Looking closely at the Listener class, I include a public data collection -- a List<double> in this case -- to collect values parsed or calculated in the listener events. You can use any data structure you like: another custom class, a list, a queue, a stack (great for calculations and expression evaluation), whatever you like.
So while the Visitor is arguably more flexible, the Listener is a strong contender too, depending on how you want to aggregate your results.
Generated accessors of parse tree context nodes do not conform getProperty()/isProperty()/hasProperty() standard. As a result, ST can’t be applied to the parse tree directly. There seems to be 3 alternatives to apply ST to the generated parse trees:
Create ST model adapter classes for each generated context node. Then ST can be applied directly to the generated parse tree. Duplicate work here is creating model adapters.
For every parse tree node create a wrapper node that conforms getProperty()/isProperty()/hasProperty() standard. Then ST can be applied to wrapper nodes. Duplicate work here is creating wrapper nodes. (In this case parse tree is not even required; auto-parse-tree construction could be turned off and wrapper (AST) nodes could be created in grammar actions).
Create a Visitor. Each visit*() instantiates an ST specific to a context node being visited, sets parameters (which could be STs returned by visiting child nodes or simple strings) and returns the ST. This is the option I’m currently using. Duplicate work here is creating visitor and assigning template parameters in code.
Is there an Antlr4 option that generates accessors of parse tree context nodes that conform getProperty()/isProperty()/hasProperty() standard? Or is there an ST4 option that allows it accessing property() instead of looking for getProperty()?
It would be nice to simply instantiate an ST template with a root context node as a parameter and let ST traverse the tree.
Just wanted to share a solution that almost avoids duplicate work while using approach #1 from my question.
Step 1: create a model adaptor that uses reflection to call a method that does not conform getProperty()/isProperty()/hasProperty() standard.
private static class MyModelAdaptor extends ObjectModelAdaptor {
#Override
public synchronized Object getProperty(Interpreter interp, ST self, Object o, Object property, String propertyName) throws STNoSuchPropertyException {
try {
return super.getProperty(interp, self, o, property, propertyName);
} catch (STNoSuchPropertyException noProperty) {
final Class<?> cls = o.getClass();
try {
return cls.getMethod(propertyName).invoke(o);
} catch (NoSuchMethodException | IllegalAccessException | InvocationTargetException e) {
throw noProperty;
}
}
}
}
Step 2: Register model adaptors
public static STGroup registerAdaptors(STGroup stg) {
final MyModelAdaptor adaptor = new MyModelAdaptor();
for (final Class<?> cls : MyParser.class.getDeclaredClasses()) {
if (isSubclassOf(cls, ParserRuleContext.class)) {
stg.registerModelAdaptor(cls, adaptor);
}
}
return stg;
}
Step 3: implement isSubclassOf() method so that registerAdaptors() compiles:
private static boolean isSubclassOf(Class<?> cls, Class<?> superCls) {
while (cls != null) {
if (cls == superCls) {
return true;
}
cls = cls.getSuperclass();
}
return false;
}
I am working on an application where I am creating a java.util.TreeMap containing data fetched from various other documents of the application and then assigning that treemap to a sessionsScope variable. This is working fine.
Now I want to provide a functionality wherein I need to store this map inside a NotesDocument.
But when I try doing this, I am getting an error.
var doc:NotesDocument = database.createDocument();
doc.replaceItemValue("Form","testForm");
print("json = "+sessionScope.get("Chart_Map"));
doc.replaceItemValue("Calender_Map",sessionScope.get("Chart_Map"));
doc.save();
Exception:
Error while executing JavaScript action expression
Script interpreter error, line=4, col=13: [TypeError] Exception occurred calling method NotesDocument.replaceItemValue(string, java.util.TreeMap) null**
Is it possible to store a java.util.TreeMap in a notesdocument field?
If yes then how to implement that?
If no then why not? has that something to do with serializability?
You can't store Java objects inside Document fields unless you use the MimeDomino Document data source
http://www.openntf.org/main.nsf/blog.xsp?permaLink=NHEF-8XLA83
Or even better the new openntf Domino API that has this functionallity built in
http://www.openntf.org/main.nsf/project.xsp?r=project/OpenNTF%20Domino%20API
using MimeStorage
Fredrik is right, the MimeDomino makes most sense. If you are not ready and your field isn't too big for a normal Notes item, you could use CustomDataBytes as Sven suggested - or you use JSON by subclassing TreeMap. It could look like this:
import java.util.TreeMap;
import java.util.Vector;
import com.google.gson.Gson;
import com.google.gson.JsonSyntaxException;
import lotus.domino.Item;
import lotus.domino.NotesException;
public class TreeMapItem extends TreeMap<String, String> {
private static final long serialVersionUID = 1L;
public static TreeMapItem load(Item source) throws JsonSyntaxException, NotesException {
Gson g = new Gson();
TreeMapItem result = g.fromJson(source.getText(), TreeMapItem.class);
return result;
}
public void save(Item target) throws NotesException {
Gson g = new Gson();
target.setValueString(g.toJson(this));
}
}
I used Google's Gson, it is quite easy, but you might need to deploy it as plug-in for the Java security to work. There is build in JSON in XPages too - a little more work. An alternate approach would be to use 2 fields in Domino, one to load the keys from and one for the values - it would be in line with Domino practises from classic.
A third approach would be be to store the values separated using a pipe character:
#SuppressWarnings({ "unchecked", "rawtypes" })
public void saveCompact(Item target) throws NotesException {
Vector v = new Vector();
for (Map.Entry<String, String> me : this.entrySet()) {
v.add(me.getKey()+"|"+me.getValue());
}
target.setValues(v);
}
#SuppressWarnings("rawtypes")
public static TreeMapItem loadCompact(Item source) throws NotesException {
TreeMapItem result = new TreeMapItem();
Vector v = source.getValues();
for (Object o : v) {
String[] candidate = o.toString().split("|");
if (candidate.length > 1) {
result.put(candidate[0], candidate[1]);
}
}
return result;
}
Let us know how it works for you