I'm trying to port working antlr to antlr4 but can't find why I get error:
syntax error: 'public' came as a complete surprise to me
on:
public root
: NEWLINE* global_block EOF -> ^(Root global_block)
;
?
btw I also get:
syntax error: '->' came as a complete surprise to me while looking for rule element
syntax error: '^' came as a complete surprise to me
Pragmatic solution
Just delete the public modifier. Rules are public by default.
Reason public is not recognized
I don't know! According to the Antlr4 grammar (https://github.com/antlr/grammars-v4/blob/master/antlr4/ANTLRv4Parser.g4), public should be allowed. However, I am not sure whether this grammar is exactly the one used for Antlr4.
A more severe problem in your spec
AST constructors are no longer supported in Antlr4 so ^(Root global_block) is not possible anymore. You need a tree visitor (or listener) to build a custom tree:
Run antlr with the -visitor option to generate a visitor class. This class is a generic class which has a visit method for each rule, e.g.,
public class PropertyFileBaseVisitor<T> extends AbstractParseTreeVisitor<T>
implements PropertyFileVisitor<T>
{
#Override public T visitFile(PropertyFileParser.FileContext ctx) { ... }
#Override public T visitProp(PropertyFileParser.PropContext ctx) { ... }
}
It returns any T, so you can simply write a visitor that builds a tree of your desired type and returns it. For a more in depth discussion of using visitors see If/else statements in ANTLR using listeners
Related
I have just discovered contextSuperClass and have been experimenting with using it to provide scope annotations when building a symbol table in a first pass (I have a forward reference DSL).
I set the option in the grammar:
options {
tokenVocab=MyLexer;
language = CSharp;
contextSuperClass = interpreter.MyParserRuleContext;
}
and I have a class that derives from ParserRuleContext:
public class MyParserRuleContext : ParserRuleContext
{
public MyParserRuleContext()
{ }
public MyParserRuleContext(ParserRuleContext parent, int invokingStateNumber) : base(parent, invokingStateNumber)
{
}
public IScope ContextScope { get; set; }
}
So far so good. I use ParseTreeWalker with a listener (Enter/Exit methods) to walk the tree for the 1st pass and build the symbol table adding local scopes, etc into my ContextScope custom property.
The first issue is of course after the symbol table pass I am at the end of the token stream - the tree is walked.
The 2nd parse uses a visitor to evaluate the final result.
I have two questions:
How do I "reset" the parser so that it is at the root again without loosing scopes I have added into my custom property?
The second question is broader, but similar. Is this even a reasonable way to add scope annotations to the parse tree?
I have previously tried to use ParseTreeProperty<IScope> to add scope annotations, but the problem is similar. During the 2nd phase, the context objects provided in the visitor are not the same objects added to ParseTreeProperty<IScope> concurrent dictionary from the 1st pass - so they are not found. Between the 1st & 2nd passes I have only found parser.reset() as a way to start the parser over, and (of course) it appears to fully reset everything and I loose the any state I created in the 1st pass.
I am likely missing completely missing something here - so any help to put me in the right direction will be greatly appreciated.
To externalize UI strings we use the "Messages-class" approach as supported e.g. in Eclipse and other IDEs. This approach requires that in each package where one needs some UI strings there has to be a class "Messages" that offers a static method String getString(key) via which one obtains the actual String to display to the user. The Strings are internally accessed/fetched using Java's Resources mechanism for i18n.
Esp. after some refactoring - we again and again have accidental imports from a class Messages from a different package.
Thus I would like to create an archunit rule checking whether we only access classes called "Messages" from the very same package. I.e. each import of a class x.y.z.Messages is an error if the package x.y.z is not the same package as the current class (i.e. the class that contains the import)
I got as far as this:
#ArchTest
void preventReferencesToMessagesOutsideCurrentPackage(JavaClasses classes) {
ArchRule rule;
rule = ArchRuleDefinition.noClasses()
.should().accessClassesThat().haveNameMatching("Messages")
.???
;
rule.check(classes);
}
but now I got stuck at the ???.
How can one phrase a condition "and the referenced/imported class "Messages" is not in the same package as this class"?
I somehow got lost with all these archunit methods of which none seems to fit here nor lend itself to compose said condition. Probably I just can't see the forest for the many trees.
Any suggestion or guidance anyone?
You need to operate on instances of JavaAccess to validate the dependencies. JavaAccess provides information about the caller and the target such that you can validate the access dynamically depending on the package name of both classes.
DescribedPredicate<JavaAccess<?>> isForeignMessageClassPredicate =
new DescribedPredicate<JavaAccess<?>>("target is a foreign message class") {
#Override
public boolean apply(JavaAccess<?> access) {
JavaClass targetClass = access.getTarget().getOwner();
if ("Message".equals(targetClass.getSimpleName())) {
JavaClass callerClass = access.getOwner().getOwner();
return !targetClass.getPackageName().equals(callerClass.getPackageName());
}
return false;
}
};
ArchRule rule =
noClasses().should().accessTargetWhere(isForeignMessageClassPredicate);
One would think that in C# 8.0 you should be able to do the following (according to this (1st snippet)):
public interface IRestApiClient : IRestClient
{
...
Task<T> PostPrivateAsync<T>(string action, OrderedDictionary<string, object> parameters = null, DeserializeCustom<T> deserializer = null)
{
return QueryPrivateAsync(Method.POST, action, parameters, deserializer);
}
...
}
public class SpecificClient : ExchangeClient, IRestApiClient, IRestHtmlClient, ISeleniumClient, IWebSocketClient
{
}
The example above won't compile because the interface members need to be explicitly and wholly implemented (including the methods supplying the default logic)
So one would think that the following should work:
public interface IRestApiClient : IRestClient
{
...
Task<T> PostPrivateAsync<T>(string action, OrderedDictionary<string, object> parameters = null, DeserializeCustom<T> deserializer = null)
{
return QueryPrivateAsync(Method.POST, action, parameters, deserializer);
}
...
}
public class SpecificClient : ExchangeClient, IRestApiClient, IRestHtmlClient, ISeleniumClient, IWebSocketClient
{
...
public async Task<T> PostPrivateAsync<T>(string action, OrderedDictionary<string, object> parameters = null, DeserializeCustom<T> deserializer = null)
=> await ((IRestApiClient) this).PostPrivateAsync(action, parameters, deserializer);
...
}
Nope, it looks like this method is recursive (despite the upcast) and will cause our favorite Stack Overflow exception.
So my question is (abstracting from the fact that I could change the design in my example), is there a way of keeping the implementation for a specific method default, preferably without the necessity of resorting to hacky or Static Helper Extension methods? I could call static extension method in both interface and the class but it kind of defeats the purpose of this feature.
// EDIT
I must admit it confuses me and it appears I am missing something critical that is obvious to other people. I didn't provide additional info because I didn't consider my issue to be code specific. Lets look at this simple example (taken from the website I linked on the beginning of my post):
According to #Panagiotis Kanavos comment: No, default members don't need to be implemented (...) what I screenshoted should not be true. Can sb please enlighten me?
// EDIT 2
As you can see I am properly targeting .NET CORE 3.0 with C# 8.0.
ERRORS:
Interface method cannot declare a body
Interface member 'void CryptoBotCoreMVC.IDefaultInterfaceMethod.DefaultMethod()' is not implemented
To answer the question in the comments: I didn't specify LangVersion explicitly in the .csproj file.
// EDIT 3
The issue was ReSharper, see:
https://stackoverflow.com/a/58614702/3783852
My comment have been deleted, presumably by the owner of the answer so I'll write it here: the clue was the fact that there was actually no error numbers, but the compilation was blocked. It turned out that there is an option to block compilation when these errors occur in ReSharper.
It seems that in the end this is a possible duplicate, but getting to this conclusion was quite a journey :).
The issue is caused by ReSharper, reference:
https://youtrack.jetbrains.com/issue/RSRP-474628
It appears that the problem will be resolved in version v2019.3 and we currently have v2019.2.3. You can setup ReSharper to block compilation depending on issue severity, the workaround is to disable this feature for the time being.
I'm writing a grammar for C++ target, however I'd like to keep it working with Java as well since ANTLR comes with great tools that work for grammars with Java target. The book ("The Definitive ANTLR 4 Reference") says that the way of achieving target independence is to use listeners and/or visitors. There is one problem though. Any predicate, local variable, custom constructor, custom token class etc. that I might need introduces target language dependence that cannot be removed, at least according to the information I took from the book. Since the book might be outdated here are the questions:
Is there a way of declaring primitive variables in language independent way, something like:
item[$bool hasAttr]
:
type ( { $hasAttr }? attr | ) ID
;
where $bool would be translated to bool in C++, but to boolean in Java (workaround would be to use int in that case but most likely not in all potential targets)
Is there a way of declaring certain code fragments to be for specific target only, something like:
parser grammar testParser;
options
{
tokenVocab=testLexer;
}
#header
<lang=Cpp>{
#include "utils/helper.h"
}
<lang=Java>{
import test.utils.THelper;
}
#members
<lang=Cpp>{
public:
testParser(antlr4::TokenStream *input, utils::THelper *helper);
private:
utils::THelper *Helper;
public:
}
<lang=Java>{
public testParser(TokenStream input, THelper helper) {
this(input);
Helper = helper;
}
private THelper Helper;
}
start
:
(
<lang=Cpp>{ Helper->OnUnitStart(this); }
<lang=Java>{ Helper.OnUnitStart(this); }
unit
<lang=Cpp>{ _localctx = Helper->OnUnitEnd(this); }
<lang=Java>{ _localctx = Helper.OnUnitEnd(this); }
)*
EOF
;
...
For the time being I'm keeping two separate grammars changing the Java one and merging the changes to C++ one once I'm happy with the results, but if possible
I'd rather keep it in one file.
This target dependency is a real nuisance and I'm thinking for a while already how to get rid of that in a good way. Haven't still found something fully usable.
What you can do is to stay with syntax that both Java and C++ can understand (e.g. write a predicate like a function call: a: { isValid() }? b c; and implement such functions in a base class from which you derive your parser (ANTLR allows to specify such a base class via the grammar option superClass).
The C++ target also got a number of additional named actions which you can use to specify C++ specific stuff only.
Since I don't know exactly what part of it alone that triggers the error, I'm not entirely sure how to better label it.
This question is a by-product of the SO question c# code seems to get optimized in an invalid way such that an object value becomes null, which I attempted to help Gary with yesterday evening. He was the one that found out that there was a problem, I've just reduced the problem to a simpler project, and want verification before I go further with it, hence this question here.
I'll post a note on Microsoft Connect if others can verify that they too get this problem, and of course I hope that either Jon, Mads or Eric will take a look at it as well :)
It involves:
3 projects, 2 of which are class libraries, one of which is a console program (this last one isn't needed to reproduce the problem, but just executing this shows the problem, whereas you need to use reflector and look at the compiled code if you don't add it)
Incomplete references and type inference
Generics
The code is available here: code repository.
I'll post a description below of how to make the projects if you rather want to get your hands dirty.
The problem exhibits itself by producing an invalid cast in a method call, before returning a simple generic list, casting it to something strange before returning it. The original code ended up with a cast to a boolean, yes, a boolean. The compiler added a cast from a List<SomeEntityObject> to a boolean, before returning the result, and the method signature said that it would return a List<SomeEntityObject>. This in turn leads to odd problems at runtime, everything from the result of the method call being considered "optimized away" (the original question), or a crash with either BadImageFormatException or InvalidProgramException or one of the similar exceptions.
During my work to reproduce this, I've seen a cast to void[], and the current version of my code now casts to a TypedReference. In one case, Reflector crashes so most likely the code was beyond hope in that case. Your mileage might vary.
Here's what to do to reproduce it:
Note: There is likely that there are more minimal forms that will reproduce the problem, but moving all the code to just one project made it go away. Removing the generics from the classes also makes the problem go away. The code below reproduces the problem each time for me, so I'm leaving it as is.
I apologize for the escaped html characters in the code below, this is Markdown playing a trick on me, if anyone knows how I can rectify it, please let me know, or just edit the question
Create a new Visual Studio 2010 solution containing a console application, for .NET 4.0
Add two new projects, both class libraries, also .NET 4.0 (I'm going to assume they're named ClassLibrary1 and ClassLibrary2)
Adjust all the projects to use the full .NET 4.0 runtime, not just the client profile
Add a reference in the console project to ClassLibrary2
Add a reference in ClassLibrary2 to ClassLibrary 1
Remove the two Class1.cs files that was added by default to the class libraries
In ClassLibrary1, add a reference to System.Runtime.Caching
Add a new file to ClassLibrary1, call it DummyCache.cs, and paste in the following code:
using System;
using System.Collections.Generic;
using System.Runtime.Caching;
namespace ClassLibrary1
{
public class DummyCache<TModel> where TModel : new()
{
public void TriggerMethod<T>()
{
}
// Try commenting this out, note that it is never called!
public void TriggerMethod<T>(T value, CacheItemPolicy policy)
{
}
public CacheItemPolicy GetDefaultCacheItemPolicy()
{
return null;
}
public CacheItemPolicy GetDefaultCacheItemPolicy(IEnumerable<string> dependentKeys, bool createInsertDependency = false)
{
return null;
}
}
}
Add a new file to ClassLibrary2, call it Dummy.cs and paste in the following code:
using System;
using System.Collections.Generic;
using ClassLibrary1;
namespace ClassLibrary2
{
public class Dummy
{
private DummyCache<Dummy> Cache { get; set; }
public void TryCommentingMeOut()
{
Cache.TriggerMethod<Dummy>();
}
public List<Dummy> GetDummies()
{
var policy = Cache.GetDefaultCacheItemPolicy();
return new List<Dummy>();
}
}
}
Paste in the following code in Program.cs in the console project:
using System;
using System.Collections.Generic;
using ClassLibrary2;
namespace ConsoleApplication23
{
class Program
{
static void Main(string[] args)
{
Dummy dummy = new Dummy();
// This will crash with InvalidProgramException
// or BadImageFormatException, or a similar exception
List<Dummy> dummies = dummy.GetDummies();
}
}
}
Build, and ensure there are no compiler errors
Now try running the program. This should crash with one of the more horrible exceptions. I've seen both InvalidProgramException and BadImageFormatException, depending on what the cast ended up as
Look at the generated code of Dummy.GetDummies in Reflector. The source code looks like this:
public List<Dummy> GetDummies()
{
var policy = Cache.GetDefaultCacheItemPolicy();
return new List<Dummy>();
}
however reflector says (for me, it might differ in which cast it chose for you, and in one case Reflector even crashed):
public List<Dummy> GetDummies()
{
List<Dummy> policy = (List<Dummy>)this.Cache.GetDefaultCacheItemPolicy();
TypedReference CS$1$0000 = (TypedReference) new List<Dummy>();
return (List<Dummy>) CS$1$0000;
}
Now, here's a couple of odd things, the above crash/invalid code aside:
Library2, which has Dummy.GetDummies, performs a call to get the default cache policy on the class from Library1. It uses type inference var policy = ..., and the result is an CacheItemPolicy object (null in the code, but type is important).
However, ClassLibrary2 does not have a reference to System.Runtime.Caching, so it should not compile.
And indeed, if you comment out the method in Dummy that is named TryCommentingMeOut, you get:
The type 'System.Runtime.Caching.CacheItemPolicy' is defined in an assembly that is not referenced. You must add a reference to assembly 'System.Runtime.Caching, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
Why having this method present makes the compiler happy I don't know, and I don't even know if this is linked to the current problem or not. Perhaps it is a second bug.
There is a similar method in DummyCache, if you restore the method in Dummy, so that the code again compiles, and then comment out the method in DummyCache that has the "Try commenting this out" comment above it, you get the same compiler error
OK, I downloaded your code and can confirm the problem as described.
I have not done any extensive tinkering with this, but when I run & reflector a Release build all seems OK (= null ref exception and clean disassembly).
Reflector (6.10.11) crashed on the Debug builds.
One more experiment: I wondered about the use of CacheItemPolicies so I replaced it with my own MyCacheItemPolicy (in a 3rd classlib) and the same BadImageFormat exception pops up.
The exception mentions : {"Bad binary signature. (Exception from HRESULT: 0x80131192)"}