I want to create the following DSL
namespace x.y.z
system <NAME> {
component A { ... }
component B { ... }
coordinator component C { ... }
component D { ... }
}
All component's have the same structure except that one and only one requires a coordinator modifier.
By now I tried two different grammars:
System:
'system' name=ValidID '{'
(coordinatorComponent=CoordinatorComponent) &
(components+=NonCoordinatorComponent*) &
(constraints=ComponentConstraints)?
'}'
;
ComponentConstraints:
'constraints' '{}'
;
CoordinatorComponent:
'coordinator' 'component' {Component}
;
NonCoordinatorComponent:
'component' {Component};
Component:
name=ValidID '{'
features+=Feature*
'}'
;
and the same one with slight changes
CoordinatorComponent:
'coordinator' {Component}
;
NonCoordinatorComponent:
{Component};
Component:
'component' name=ValidID '{'
features+=Feature*
'}'
;
The first one results in an error rule ruleSystem failed predicate: {getUnorderedGroupHelper().canLeave(grammarAccess.getSystemAccess().getUnorderedGroup())}? in the editor when writing the DSL (not the grammar).
The second one works but still I think it is weird because it is not really using a modifier but a whole new type. Is there a better way to define Component in general and using a modifier which MUST be used exactly once within System?
what about using parser fragments
System:
'system' name=ValidID '{'
((coordinatorComponent=CoordinatorComponent) &
(components+=NonCoordinatorComponent*) &
(constraints=ComponentConstraints)?)
'}'
;
ComponentConstraints:
'constraints' '{}'
;
CoordinatorComponent:
'coordinator' 'component' Component
;
NonCoordinatorComponent:
'component' Component;
fragment Component:
name=ValidID '{'
features+=Feature*
'}'
;
Related
I am declaring my error listener like this:
class GeneratorErrorListener(ErrorListener):
def __init__(self, listener):
super().__init__()
self.listener = listener
def systaxError(self, recognizer, offendingSymbol, line, col, msg, e):
log_it("Syntax error at line {} col {}: {}".format(line, col, msg))
I am not yet making use of the listener passed in, but will when I get it working.
and setting it up like this:
...
# Set up new error listener
parser.removeErrorListeners()
parser.addErrorListener(GeneratorErrorListener(listener))
tree = parser.protocol()
...
walker.walk(listener, tree)
Then I am testing it with some input that has a syntax error (AFAICS):
The grammar fragment is:
enumEltDecl : INT '=' ID ( ':' STRING)?
| 'default' '=' STRING
;
enumDecl: 'enum' ID ( ':' ID )? '{' enumEltDecl (',' enumEltDecl )*
(',')? '}' ;
and I can parse those things fine. However, the following input which I think should be a syntax error, and does cause parsing to stop, does not invoke the error listener:
emum some_emum:uint8 {
};
It should have at least one enumEltDecl.
Any thoughts on what I have done wrong? I have looked at the runtime code for the ErrorListener class and it seems straightforward.
More Information
The code is here: https://gitlab.com/realrichardsharpe/wireshark-generator-python
Use the following steps to see the issue:
cd src
./GenTool.py -t C ../test-data/syntax-error.proto
You will see the following output:
#include "config.h"
#include <epan/packet.h>
#include <epan/expert.h>
//Generating code for enum cmd_enum
enum cmd_enum {
CMD1 = 0x14;
CMD2 = 0x15;
CMD3 = 0x28;
CMD4 = 0x29;
CMD5 = 0x3C;
CMD5 = 0x3D;
};
//We have a uint8
static const range_string cmd_enum_rvals[] = {
{ 0, 19, "Reserved", }
{ 0x14, 0x14, "cmd1" },
{ 0x15, 0x15, "cmd2" },
{ 22, 39, "Reserved", }
{ 0x28, 0x28, "cmd3" },
{ 0x29, 0x29, "cmd4" },
{ 42, 59, "Reserved", }
{ 0x3C, 0x3C, "cmd5" },
{ 0x3D, 0x3D, "cmd6" },
{ 62. 255, "Reserved" },
};
And it stops without my ErrorListener being called. The ErrorListener is in GenTool.py.
Strangely, with a little rearrangement of the code and after lots of debugging it now seems to be working because I get the following errors with a different set of input data:
./GenTool.py -t C xxx.proto
line 3:0 mismatched input '}' expecting {'default', INT}
line 4:0 missing ';' at '<EOF>'
Syntax error on line 1
The first two lines are generated by my error listener.
UPDATE: With a little comparison, I discovered that my test case had a symbol in it that is not recognized by the grammar and things went off the rails at that point.
The real problem was that my grammar was incorrect. The first line should have been:
protocol : protoDecl+ EOF ;
In my original grammer the EOF was missing which caused the parser to stop when it hit something that did not match the grammer.
I found the code (reproduced below) in an article from Terrence Parr showing how INCLUDE files could be handled in ANTLR3 for Java. I tried to add this to a grammar I use with ANTLR4 (with a C++ target) but when I tried to generate a parser, I got the errors
error(50): : syntax error: '^' came as a complete surprise to me
error(50): : syntax error: mismatched input '->' expecting SEMI while matching a rule
error(50): : syntax error: '^' came as a complete surprise to me
error(50): : syntax error: '(' came as a complete surprise to me while matching rule preamble
and I have no idea what these error means. Can anyone explain and perhaps show me the way forward?
(NB: I'm not wild about polluting the grammar file with code, I'm using the visitor pattern but I'll take it if I can!)
Thanks
include_filename :
('a'..'z' | 'A'..'Z' | '.' | '_')+
;
include_statement
#init { CommonTree includetree = null; }
:
'include' include_filename ';' {
try {
CharStream inputstream = null;
inputstream = new ANTLRFileStream($include_filename.text);
gramLexer innerlexer = new gramLexer(inputstream);
gramParser innerparser = new gramParser(new CommonTokenStream(innerlexer));
includetree = (CommonTree)(innerparser.program().getTree());
} catch (Exception fnf) {
;
}
}
-> ^('include' include_filename ^({includetree}))
;
Starting with ANTLR4 it is no longer possible to manipulate the generated parse tree with grammar rules. In fact ANTLR3 generated an AST (abstract syntax tree), which is a subset of a parse tree (as generated by ANTLR4). That in turn means you cannot keep the tree rewrite syntax (the part starting with ->). Hence you should change the code to:
include_statement
#init { CommonTree includetree = null; }
:
'include' Include_filename ';' {
try {
CharStream inputstream = null;
inputstream = new ANTLRFileStream($include_filename.text);
gramLexer innerlexer = new gramLexer(inputstream);
gramParser innerparser = new gramParser(new CommonTokenStream(innerlexer));
includetree = (CommonTree)(innerparser.program().getTree());
} catch (Exception fnf) {
;
}
}
;
I am trying to learn to write puppet modules in a good way, so I've started looking around for tutorials and howto.
I've seen that users suggest writing the main class in the following way, but It's actually failing for me.
I am honestly a bit confused how the 2 blocks between brackets are actually connected, and so I might be not seeing an obvious error or real missing comma.
I am running Puppet 3.8 by the way
class icinga2 {
$version = 'present'
$enable = true
$start = true
} {
class{'icinga2::install': } ->
class{'icinga2::config': } ~>
class{'icinga2::service': } ->
Class["icinga2"]
}
Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Syntax error at '{'; expected '}' at /etc/puppet/modules/icinga2/manifests/init.pp:5
Your problem here is that your parameters must be surrounded by (), not {}. Also, they should be commas separated.
class icinga2 (
$version = 'present',
$enable = true,
$start = true,
) {
class{'icinga2::install': } ->
class{'icinga2::config': } ~>
class{'icinga2::service': } ->
Class["icinga2"]
}
I am working on a code generator for my grammar that I have created:
Domainmodel:
(elements+=AbstractElement)*;
PackageDeclaration:
'package' name=QualifiedName '{'
(elements+=AbstractElement)*
'}';
AbstractElement:
PackageDeclaration | Type | Import;
QualifiedName:
ID ('-' ID)*;
QualifiedDate:
INT('-' INT)*
;
Import:
'import' importedNamespace=QualifiedNameWithWildcard;
QualifiedNameWithWildcard:
QualifiedName '.*'?;
Type:
(data+= DataType)* man+=Entity ;
DataType:
'tag' name=Tag;
Tag:
Hobbies='hobbies' | Work= 'work' |Fun='fun'
;
Entity:
name=Category '{'
feature+=Feature*
'}'
;
Feature:
component+=Man(',' component+=Opt)*
;
enum Category:
Blog='blog' | Article='articles'
;
Man:
name='title' '=' type=QualifiedName
;
Opt:
Tags|Date
;
Tags:
name='tags' '=' '['type= Tag(','tag+=Tag)*']'
|
name='tags' '=' '[' ']'
;
Date:
name='date' '=' type=QualifiedDate
;
I want my output of my code generator to look like this:
---
layout: post
title: "My Trip"
categories: blog
excerpt:
tags: [fun,hobbies]
image:
feature:
date: 2016-06-01T14:19:19-04:00
modified:
---
All I can get right is the static text, I can't seem to call: Category , title , tags , date
I've been trying for so long now but I can't seem to get anywhere, I keep getting strange errors which i don't understand
One of my attempts for just seeing what I can generate is:
class MyDslGenerator implements IGenerator2 {
def compile(Entity e)
{
'''
---
layout: post
title: "My Trip"
categories:«e.name»
excerpt:
tags: [fun,hobbies]
image:
feature:
date: 2016-06-01T14:19:19-04:00
modified:
---
'''
}
override doGenerate(Resource input, IFileSystemAccess2 fsa, IGeneratorContext context) {
for (e : input.allContents.toIterable.filter(Entity)) {
fsa.generateFile(
e.generateName,
e.compile)
}
}
when I run the generator I don't get anything replaced by <>. I can't seem to figure it out.
is this a question on how to walk the AST. your grammar and thus the inferred metamodel is quite "bad" to walk so you may have to do something like
title: «(e.feature.head.component.head as Man).type»
so i recommend you to restructure your grammar/AST to fit the stuff you need.
you can set the encoding for the xtend/xtext plugin like this
tasks.withType(org.xtext.gradle.tasks.XtextGenerate) {
options.encoding = 'ISO-8859-1'
}
does that help?
/**
* generated by Xtext 2.10.0
*/
package org.xtext.example.mydsl.tests;
import com.google.inject.Inject;
import org.eclipse.xtend2.lib.StringConcatenation;
import org.eclipse.xtext.junit4.InjectWith;
import org.eclipse.xtext.junit4.XtextRunner;
import org.eclipse.xtext.junit4.util.ParseHelper;
import org.eclipse.xtext.xbase.lib.Exceptions;
import org.junit.Assert;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.xtext.example.mydsl.myDsl.Domainmodel;
import org.xtext.example.mydsl.tests.MyDslInjectorProvider;
#RunWith(XtextRunner.class)
#InjectWith(MyDslInjectorProvider.class)
#SuppressWarnings("all")
public class MyDslParsingTest {
#Inject
private ParseHelper<Domainmodel> parseHelper;
#Test
public void loadModel() {
try {
StringConcatenation _builder = new StringConcatenation();
_builder.append("Hello Xtext!");
_builder.newLine();
final Domainmodel result = this.parseHelper.parse(_builder);
Assert.assertNotNull(result);
} catch (Throwable _e) {
throw Exceptions.sneakyThrow(_e);
}
}
}
I have a flex bison application. For a few of my tokens, I copy out the yytext from flex using strdup. This works great except when there is an error of an unexpected token.
simple example
flex.l:
...
[a-zA-Z0-9]+ { lval.string = strdup(yytext); return IDENT };
[\{\}] { return yytext[0] };
...
and
parse.y
...
%destructor { free($$); } IDENT
%destructor { free($$->name); free($$->type); free($$); } tag
...
tag: IDENT '{' IDENT '}'
{
struct tag *mytag = malloc(sizeof(struct tag));
mytag->name = $1;
mytag->type = $3;
$<tag>$ = mytag;
}
...
Now suppose I hand it the input:
blah blah blah
The lexer will send up the first IDENT token, which gets pushed onto the stack. After the first token it's expecting a bracket token, but instead gets another IDENT token. This is a syntax error. The destructor will be called on the first IDENT token, but not on the second one (the unexpected one). I haven't been able to find a way to destruct the unexpected token. Does anyone know how I should do it?
I found that appropriate use of the 'error' token in flex prompts it to correctly call the destructor function. Go me!
parse.y
...
%destructor { free($$); } IDENT
%destructor { free($$->name); free($$->type); free($$); } tag
...
tags: tag tags | error tags | ;
tag: IDENT '{' IDENT '}'
{
struct tag *mytag = malloc(sizeof(struct tag));
mytag->name = $1;
mytag->type = $3;
$<tag>$ = mytag;
}
...