I generated a c++ python parser with antlr4 c++ target but when I try to use it I have the following error:
Python3Lexer.h:48:5: error: stray ‘#’ in program
This Python3Lexer.h (generated with Antlr4 c++ target) does not look good?
The error line is #Override which is a java keyword, not c++!
Do you know what I am doing wrong?
Here is what this Python3Lexer.h looks like:
#include "antlr4-runtime.h"
class Python3Lexer : public antlr4::Lexer {
public:
(...)
Python3Lexer(antlr4::CharStream *input);
~Python3Lexer();
// A queue where extra tokens are pushed on (see the NEWLINE lexer rule).
private java.util.LinkedList<Token> tokens = new java.util.LinkedList<>();
// The stack that keeps track of the indentation level.
private java.util.Stack<Integer> indents = new java.util.Stack<>();
// The amount of opened braces, brackets and parenthesis.
private int opened = 0;
// The most recently produced token.
private Token lastToken = null;
#Override
public void emit(Token t) {
super.setToken(t);
tokens.offer(t);
}
#Override
public Token nextToken() {
(...)
If you look inside the grammar file you're using (which you've linked in a comment), you'll see that it contains Java code. In order to use this grammar in C++, you'll first have to translate that Java code to C++.
I want to write a groovy DSL with syntax:
returnValue when booleanCondition
I want to use compilation customizers to transform this expression to a typical if return statement using AST transformations.
For this script:
2 when 1 == 1
I get exception message:
MultipleCompilationErrorsException: startup failed:
Script1.groovy: 1: expecting EOF, found '1' # line 1, column 8.
I don't understand why my compilation customizer is not called at all?
I need it to be called before compilation so I can make it into a valid groovy code.
If the script contains valid groovy code, then my compilation customizer is called.
My code:
class MyDslTest {
public static void main(String[] args) {
String script = '''2 when 1 == 1
'''
def compilerConfig = new CompilerConfiguration()
compilerConfig.addCompilationCustomizers(new MyCompilationCustomizer())
GroovyShell groovyShell = new GroovyShell(compilerConfig)
groovyShell.evaluate(script)
}
}
class MyCompilationCustomizer extends CompilationCustomizer {
MyCompilationCustomizer() {
super(CompilePhase.CONVERSION)
}
#Override
void call(SourceUnit source, GeneratorContext context, ClassNode classNode) throws CompilationFailedException {
println 'in compilation customizer'
}
}
The problem is that your code is not syntactically valid. A compilation customizer will not workaround that: to be able to get an AST, on which the customizer will work, you have to produce syntactically correct code. One option is to use a different AntlrParserPlugin, but in general I don't recommend to do it because it will modify the sources before parsing, and therefore create a mismatch between the AST and the actual source.
I've created my project in vs2008.It works fine.But when i opened the solution and try to build it in vs2012 i am getting the following error in TransactionDB.dbml page.
a partial method may not have multiple defining declarations
What could be the problem??
.net supports partial methods.
It means you write a definition in one part of the partial class and the implementation in another. Like this:
partial class MyClass
{
partial void MyPartialMethod(string s);
}
// This part can be in a separate file.
partial class MyClass
{
// Comment out this method and the program will still compile.
partial void MyPartialMethod(string s)
{
Console.WriteLine("Something happened: {0}", s);
}
}
In your case I you have the two definitions of the partial method causing the compiler to fail.
Source MSDN
The defining declaration of a partial method is the part that specifies the method signature, but not the implementation (method body). A partial method must have exactly one defining declaration for each unique signature. Each overloaded version of a partial method must have its own defining declaration.
To correct this error
Remove all except one defining declaration for the partial method.
Example
// cs0756.cs
using System;
public partial class C
{
partial void Part();
partial void Part(); // CS0756
public static int Main()
{
return 1;
}
}
class foo
{
public static List parse(String input_xml)
{
try {
// JAXB context created.
JAXBContext jaxbContext = JAXBContext.newInstance(Root.class);
ByteArrayInputStream input = new ByteArrayInputStream(input_xml.getBytes());
Unmarshaller jaxbUnmarshaller = jaxbContext.createUnmarshaller();
// Error comes while unmarshalling the input
Root root = (Root) jaxbUnmarshaller.unmarshal(input);
HashMap<String,String> map1 = new HashMap<String,String>();
//... Code to retrieve values from "root" and put it in list
}
}
public static void main(String[] args)
{
String input= "<?xml version=\"1.0\"?>"
+"<root>"
+"<attribs R:SSB:12 = \"ABC\"/>" +
"</root>";
List list = parse(input);
}
}
"attribs" can contains multiple "attributes-value" pairs, hence i need to "UNMARSHAL" it using Jaxb.
I tried using QName , but its throwing error.
Question : While parsing , I am getting this error : "An invalid second ':' was found in the element type or attribute name." .... But i couldn't find any help .... attribute name and values are dynamic in my case.
Thanks for any help!!!*/
The problem is that you are trying to feed JAXB with invalid XML:
<attribs R:SSB:12 = "ABC">
Colon symbol has special meaning in XML: it separates namespace and attribute (or tag) name under that namespace. That's why attribute (or tag) can only contain one colon.
If your attribute names are coming from elsewhere, you have to escape colons to make them valid XML attributes. Try replacing colons with dash symbol (-), for example:
<attribs R-SSB-12="ABC">
The extra : is causing problems as by default JAXB expects the portion behind the : is the namespace prefix. You can solve this by forcing JAXB to use a parser that is not namespace aware.
This can easily be done with a SAX parser and leveraging JAXB's UnmarshallerHandler as the ContentHandler.
Full Example
java xml annotation get field with namespace, <aaa:bbb>value</aaa:bbb>
How can I make a rule to match all of its alternatives only once in any order, in ANTLR?
i.e.
rule: ('example\r\n' | 'example2\r\n') nextRule
I would like 'example' and 'example2' to match only once before moving onto the next rule.
Should match inputs of:
example
example2
or
example2
example
but not inputs of:
example
example
example2
How can I make a rule to match all of its alternatives only once in any order, in ANTLR?
"All of its alternatives only once" is simply rule: altA altB altC .... That's the easy part. Asking rule to accept all alternatives altA, altB, altC, etc. in any arrangement means accommodating every arrangement. That's easy enough to handle manually for small numbers (rule: (altA altB | altB altA);). But there's no automatic shortcut that I'm aware of to handling all the permutations for you automatically.
Here are some approaches in case there isn't a built-in way and assuming you can't relax your requirements. Caveats: I don't know the full scope of your problem; I don't know your grammar; I don't know why you want what you're asking for; I don't know what class of solution you prefer, other than you'd probably like it easier than any of these options.
First, you could bite the bullet and produce all the permutations of matches yourself, either by hand or by running a permutation generator. Then ANTLR would have what you want in a manner that it understands. It's crude but efficient: it's straight ANTLR syntax so there's no external code involved as there is in the options below.
For example, assume you have a rule field that processes input like "public static final x", with all three modifiers expected but in no particular order. The permutations would look like so:
field : modifiers ID EOF;
modifiers
: PUBLIC STATIC FINAL //ABC
| PUBLIC FINAL STATIC //ACB
| STATIC PUBLIC FINAL //BAC
| STATIC FINAL PUBLIC //BCA
| FINAL PUBLIC STATIC //CAB
| FINAL STATIC PUBLIC //CBA
;
That's the end of it. No external code, no predicates, no nothing.
Second, you could use semantic predicates in the grammar to ensure that all matches are provided and matched without duplicates. There are various ways of writing the predicates themselves, but it boils down to tracking what matches have been made (to prevent duplicates) and then testing whether the rule has matched all the parts that it expects. Here is a basic example, following the same requirements as the previous one:
field
locals [java.util.HashSet<String> names = new java.util.HashSet<String>();]
: modifiers ID EOF;
modifiers
//Ensure that the full number of modifiers have been provided
: {$field::names.size() < 3}? modifier modifiers
| {$field::names.size() == 3}? //match nothing once we have (any) three modifiers
;
modifier
//Ensure that no duplicates have been provided
: {!$field::names.contains("public")}? PUBLIC {$field::names.add("public");}
| {!$field::names.contains("static")}? STATIC {$field::names.add("static");}
| {!$field::names.contains("final")}? FINAL {$field::names.add("final");}
;
Here rule field tracks the modifier names in local variable names. Rule modifiers calls rule modifier until names contains three values. Rule modifier matches any name that doesn't have a corresponding key in names. Note that the values are manually added to names. They could be any arbitrary value as long as modifier's alternatives add the same value to both sides of the token it's matching.
My implementation is a little crude because the modifiers end up nesting in the produced parse tree (since modifiers contains one modifier and one modifiers that contains one modifier and one modifiers that...), but I hope you get the idea.
Third, you could leave the poor parser alone and test for completeness in the calling code. This can be done during parsing using a parser listener or done after parsing using the ParserRuleContext object produced by the parser. This breaks the problem into two parts: let the parser solve "any X, Y, Z in any order" and let the calling code solve "all and only just X, Y, Z."
Here is an example using the listener approach:
//partial grammar
field : modifier* ID EOF; //accept any modifiers in any order
modifier
: PUBLIC
| STATIC
| FINAL
;
//snippet of calling code
//initialize lexer and parser
parser.addParseListener(new MyGrammarBaseListener() {
#Override
public void exitField(FieldContext ctx) {
// The field rule has finished. Let's verify that no modifiers
// were duplicated.
//TODO check for duplicates, ensure all modifiers are specified.
//TODO log errors accordingly.
}
});
//Call the parser.
parser.field();
The grammar is kept clean. Modifiers can appear in the input arbitrarily before ID, in any number and in any order. The calling code performs the tests by whatever means it chooses, logging whatever errors it wants.
Here is an uberexample that pulls together the three options I mentioned, to give a clearer idea of what I'm talking about.
Modifiers.g
grammar Modifiers;
//Hard-coded version : all the permutations are specified //
permutationField : permutationModifiers ID EOF;
permutationModifiers
: PUBLIC STATIC FINAL //ABC
| PUBLIC FINAL STATIC //ACB
| STATIC PUBLIC FINAL //BAC
| STATIC FINAL PUBLIC //BCA
| FINAL PUBLIC STATIC //CAB
| FINAL STATIC PUBLIC //CBA
;
// Predicate version : use semantic predicates to prevent duplicates and ensure all the modifiers are provided //
predicateField
locals [java.util.HashSet<String> names = new java.util.HashSet<String>();]
: predicateModifiers ID EOF;
predicateModifiers
//Ensure that the full number of modifiers have been provided
: {$predicateField::names.size() < 3}? predicateModifier predicateModifiers
| {$predicateField::names.size() == 3}? //match nothing once we have (any) three modifiers
;
predicateModifier
//Ensure that no duplicates have been provided
: {!$predicateField::names.contains("public")}? PUBLIC {$predicateField::names.add("public");}
| {!$predicateField::names.contains("static")}? STATIC {$predicateField::names.add("static");}
| {!$predicateField::names.contains("final")}? FINAL {$predicateField::names.add("final");}
;
//Listener version : test everything when the parser calls the listener //
listenerField : listenerModifier* ID EOF;
listenerModifier
: PUBLIC
| STATIC
| FINAL
;
PUBLIC : 'public';
STATIC : 'static';
FINAL : 'final';
FOO : 'foo';
ID : [a-zA-Z]+;
WS : [ \r\n\t]+ -> skip;
ModifiersTest.java
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import org.antlr.v4.runtime.ANTLRInputStream;
import org.antlr.v4.runtime.BaseErrorListener;
import org.antlr.v4.runtime.CommonTokenStream;
import org.antlr.v4.runtime.RecognitionException;
import org.antlr.v4.runtime.Recognizer;
import org.antlr.v4.runtime.misc.Nullable;
public class ModifiersTest {
public static void main(String[] args) {
run("public static final x", "ok");
run("final static public x", "ok");
run("static public static final x", "too many modifiers");
run("static x", "missing modifiers");
run("final final x", "missing & duplicated modifiers");
}
private static void run(String code, String title) {
System.out.printf("%n---%n**Input : %s**%n%n\t%s%n%n", title, code);
System.out.println("**Permutation Output**\n");
runPermutationTest(code);
System.out.println();
System.out.println("**Predicate Output**\n");
runPredicateTest(code);
System.out.println();
System.out.println("**Listener Output**\n");
runListenerTest(code);
System.out.println();
}
private static void runPermutationTest(String code) {
ModifiersParser parser = createParser(code);
parser.permutationField();
System.out.println("\t(done)");
}
private static void runPredicateTest(String code) {
ModifiersParser parser = createParser(code);
parser.predicateField();
System.out.println("\t(done)");
}
private static void runListenerTest(String code) {
ModifiersParser parser = createParser(code);
parser.addParseListener(new ModifiersBaseListener() {
#Override
public void exitListenerField(ModifiersParser.ListenerFieldContext ctx) {
// The field rule has finished. Let's verify that no modifiers
// were duplicated.
HashSet<String> uniqueNames = new HashSet<String>();
ArrayList<String> allNames = new ArrayList<String>();
HashSet<String> expectedNames = new HashSet<String>();
expectedNames.add("public");
expectedNames.add("static");
expectedNames.add("final");
if (ctx.listenerModifier() != null && !ctx.listenerModifier().isEmpty()) {
List<ModifiersParser.ListenerModifierContext> modifiers = ctx.listenerModifier();
// Collect all the modifier names in a set.
for (ModifiersParser.ListenerModifierContext modifier : modifiers) {
uniqueNames.add(modifier.getText());
allNames.add(modifier.getText());
}
}
// Is the number of unique modifiers less than the number of
// all given modifiers? If so, then there must be duplicates.
if (uniqueNames.size() < allNames.size()) {
ArrayList<String> names = new ArrayList<String>(allNames);
for (String name : uniqueNames){
names.remove(name);
}
System.out.println("\tDetected duplicate modifiers : " + names);
} else {
System.out.println("\t(No duplicate modifiers detected)");
}
//Are we missing any expected modifiers?
if (!uniqueNames.containsAll(expectedNames)) {
ArrayList<String> names = new ArrayList<String>(expectedNames);
names.removeAll(uniqueNames);
System.out.println("\tDetected missing modifiers : " + names);
} else {
System.out.println("\t(No missing modifiers detected)");
}
}
});
parser.listenerField();
System.out.println("\t(done)");
}
private static ModifiersParser createParser(String code) {
ANTLRInputStream input = new ANTLRInputStream(code);
ModifiersLexer lexer = new ModifiersLexer(input);
ModifiersParser parser = new ModifiersParser(new CommonTokenStream(lexer));
BaseErrorListener errorListener = createErrorListener();
lexer.addErrorListener(errorListener);
parser.addErrorListener(errorListener);
return parser;
}
private static BaseErrorListener createErrorListener() {
BaseErrorListener errorListener = new BaseErrorListener() {
#Override
public void syntaxError(Recognizer<?, ?> recognizer, #Nullable Object offendingSymbol, int line,
int charPositionInLine, String msg, #Nullable RecognitionException e) {
//Print the syntax error
System.out.printf("\t%s at (%d, %d)%n", msg, line, charPositionInLine);
}
};
return errorListener;
}
}
Test Scenarios (output from the code above)
Input : ok
public static final x
Permutation Output
(done)
Predicate Output
(done)
Listener Output
(No duplicate modifiers detected)
(No missing modifiers detected)
(done)
Input : ok
final static public x
Permutation Output
(done)
Predicate Output
(done)
Listener Output
(No duplicate modifiers detected)
(No missing modifiers detected)
(done)
Input : too many modifiers
static public static final x
Permutation Output
extraneous input 'static' expecting 'final' at (1, 14)
(done)
Predicate Output
no viable alternative at input 'static' at (1, 14)
(done)
Listener Output
Detected duplicate modifiers : [static]
(No missing modifiers detected)
(done)
Input : missing modifiers
static x
Permutation Output
no viable alternative at input 'staticx' at (1, 7)
(done)
Predicate Output
no viable alternative at input 'x' at (1, 7)
(done)
Listener Output
(No duplicate modifiers detected)
Detected missing modifiers : [final, public]
(done)
Input : missing & duplicated modifiers
final final x
Permutation Output
no viable alternative at input 'finalfinal' at (1, 6)
(done)
Predicate Output
no viable alternative at input 'final' at (1, 6)
(done)
Listener Output
Detected duplicate modifiers : [final]
Detected missing modifiers : [static, public]
(done)
With ANTLR 4, I would actually prefer to use something like the following, where the input is expected to contain exactly 1 each of a, b, and c.
items : (a | b | c)*;
Then in a listener, I would use code like the following:
#Override
public void enterItems(ItemsContext ctx) {
if (ctx.a().size() != 1) {
// report error
} else if (ctx.b().size() != 1) {
// report error
} else ...
}