Handle a bunch of arbitrary literals using Groovy AST Transformation - groovy

I read a lot about AST Transformation these days and now I want to handle some arbitrary literals not known by groovy. The specific idea is to enable groovy handling plain sql.
If you write select a from tab where b = 'x' in GroovyConsole, the AST looks like this at some point
MethodCall - this.select(a).from(tab).where((b = x))
With some effort, it should be possible to turn this into a sql-statement.
So everything is fine as long as I don't use an asterisk.
If I write select * from tab no AST can be built (no matter what phase) and an error occures:
Unable to produce AST for this phase due to earlier compilation error:
startup failed:
script123.groovy: 1: expecting EOF, found 'tab' # line1, column 15.
So here's the question:
How can I turn the asterisk into something processable?

I tried to get the CST after parsing was done. But it's always null and I couldn't find out why.
But I found another approach:
http://java.dzone.com/articles/run-your-antlr-dsl-groovy

Related

Call groovy script dynamically in Apache Camel using doTry-doCatch

I'm building a route which calls a groovy script whose path is dynamically computed and, if the script can't be found, defaults to a generic, static script:
.doTry()
.toD("language://groovy:resource:classpath:scripts/${exchangeProperty.consumerType}ResponseHandler.groovy")
.doCatch(FileNotFoundException.class)
.script().groovy("resource:classpath:scripts/defaultResponseHandler.groovy")
.end()
The problem is that the exchange property consumerType is not resolved since the uri string parameter of toD is evaluated using groovy and not simple.
MultipleCompilationErrorsException -> startup failed:
Script_09b4150584d9e2c979353feee06897b5.groovy: 1: Unexpected input: 'scripts/${exchangeProperty.consumerType}' # line 1, column 20.
resource:classpath:scripts/${exchangeProperty.consumerType}ResponseHandler.groovy
^
1 error
How can I obtain the desired behavior?
According to the error shown there, it seems Camel is not able to resolve the string you provided in the toD().
By default, the expression you pass to a dynamic to is evaluated as Simple language but, as described in To Dynamic Camel documentation, you can specify other languages for the dynamic evaluation.
In your case, you are trying to evaluate the endpoint with groovy language but then you're using Simple language to substitute a piece of the name of the script.
One solution I've found (yet not the best) would be to specify the language for the interpretation of the string as simple and then use language:groovy to specify the endpoint that will need to be called.
You could write something like this:
.doTry()
.toD("language:simple:language://groovy:resource:classpath:scripts/${exchangeProperty.consumerType}ResponseHandler.groovy")
.doCatch(FileNotFoundException.class)
.script().groovy("resource:classpath:scripts/defaultResponseHandler.groovy")
.end()
It seems to work, but I hope someone else comes up with a better solution.

What $() syntax means for Groovy language?

I found this in Groovy Syntax documentation at 4.6.1. Special cases:
As slashy strings were mostly designed to make regexp easier so a few
things that are errors in GStrings like $() or $5 will work with
slashy strings.
What $() syntax means? give some usage examples please
I also found it at Define the Contract Locally in the Repository of the Fraud Detection Service:
body([ // (4)
"client.id": $(regex('[0-9]{10}')),
loanAmount : 99999
])
but I don't understand what $() means when used with regex('[0-9]{10}').
It means nothing (or what you make of it). There are two places, you
are addressing, but they have nothing to do with each other.
The docs just mention this as "you can use slashy strings to write
things, that would give you an error with a GString" - the same is true
for just using '-Strings.
E.g.
"hello $()"
Gives this error:
unknown recognition error type: groovyjarjarantlr4.v4.runtime.LexerNoViableAltException
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
/tmp/x.groovy: 1: token recognition error at: '(' # line 1, column 9.
"hello $()"
The parser either wants a { or any char, that is a valid first char
for a variable (neither ( nor 5 is).
The other place you encountered $() (in Spring cloud contract), this
is just a function with the name $.
Form the docs 8. Contract DSL:
You can set the properties inside the body either with the value method or, if you use the Groovy map notation, with $()
So this is just a function, with a very short name.
E.g. you can try this yourself:
void $(x) { println x }
$("Hello")

What to do when a code review tool declares unmatched types?

I am working on developing a large-scale Python (backend) project. I was working with a firm that does extensive testing, and they built the frontend and test tools. Before every deploy, all the tools (like linters) are run regularly.
I had put down the code for a while, and now it fails many tests. Some of these are deprecation warnings for features or syntax soon to be deprecated, and they note they started classifying those as warnings (to later become errors) starting January 1, 2020, so I know they make dynamic changes in the tools themselves.
My problem is a bunch of code that used to pass no longer does. And the error is always the same: if I have a line that looks like so, I get an error that says something along the lines of "error: may not use operator '-' with incompatible types; a and b are of types numpy.array and NoneType":
x = a - b
This gets fixed by making the code super-messy with this sort of fix:
x = a.astype(float) - b.astype(float)
It's even worse because in the actual code there are 3 variables, all doing addition and subtraction with a 'c' that is an integer array kicking around along with the two numpy arrays. But then the code goes from:
x = a - b - c
to:
x = a.astype(float) - b.astype(float) - c.astype(float)
And this won't work since int's don't have an astype method. The error looks like this now:
File "/home/engine.py", line 165, in Foo
lower_array[t].astype(float)) / num_values.astype(float)
AttributeError: 'NoneType' object has no attribute 'astype'
Thus, I end up with:
x = a.astype(float) - b.astype(float) - float(c)
This is all extraordinarily cumbersome and nasty casting that is required, and makes the code impossible to read.
The odd thing to me is that all three arrays were instantiated as numpy arrays, i.e.,:
a=numpy.array(_a)
b=numpy.array(_b)
c=numpy.array(_c)
When I ask the code to put output to stdout the type of all three vars, they all say . Yet, the next line of code blows up and dumps, saying "Attribute error: 'NoneType' object has no attribute 'astype'"
I can't fathom how a static code analyzer determines the types - other than as numpy.ndarray type - since Python uses duck-typing. Thus, the type could change dynamically. But that's not the case here; all three vars are identified as numpy.ndarray type, but "z = a - b - c" fails.
Anyone understand what's going on here?
After much work, the answer is to ignore the linter. Readable code is the object, not code that satisfies a linter.

squeak(smalltallk) how to 'inject' string into string

I'm writing a class named "MyObject".
one of the class methods is:
addTo: aCodeString assertType: aTypeCollection
when the method is called with aCodeString, I want to add (in runtime) a new method to "MyObject" class which aCodeString is it's source code and inject type checking code into the source code.
for example, if I call addTo: assertType: like that:
a := MyObject new.
a addTo: 'foo: a boo:b baz: c
^(a*b+c)'
assertType: #(SmallInteger SmallInteger SmallInteger).
I expect that I could write later:
answer := (a foo: 2 boo: 5 baz: 10).
and get 20 in answer.
and if I write:
a foo: 'someString' boo: 5 baz: 10.
I get the proper message because 'someString' is not a SmallInteger.
I know how to write the type checking code, and I know that to add the method to the class in runtime I can use 'compile' method from Behavior class.
the problem is that I want to add the type checking code inside the source code.
I'm not really familiar with all of squeak classes so I'm not sure if I rather edit the aCodeString as a string inside addTo: assertType: and then use compile: (and I don't know how to do so), or that there is a way to inject code to an existing method in Behavior class or other squeak class.
so basically, what I'm asking is how can I inject string into an existing string or to inject code into an existing method.
There are many ways you could achieve such type checking...
The one you propose is to modify the source code (a String) so as to insert additional pre-condition type checks.
The key point with this approach is that you will have to insert the type checking at the right place. That means somehow parsing the original source (or at least the selector and arguments) so as to find its exact span (and the argument names).
See method initPattern:return: in Parser and its senders. You will find quite low level (not most beautiful) code that feed the block (passed thru return: keyword) with sap an Array of 3 objects: the method selector, the method arguments and the method precedence (a code telling if the method is connected to unary, binary or keyword message). From there, you'll get enough material for achieving source code manipulation (insert a string into another with copyReplace:from:to:with:).
Do not hesitate to write small snippets of code and execute in the Debugger (select code to debug, then use debug it menu or ALT+Shift+D). Also use the inspectors extensively to gain more insight on how things work!
Another solution is to parse the whole Abstract Syntax Tree (AST) of the source code, and manipulate that AST to insert the type checks. Normally, the Parser builds the AST, so observe how it works. From the modified AST, you can then generate new CompiledMethod (the bytecode instructions) and install it in methodDictionary - see the source code of compile: and follow the message sent until you discover generateMethodFromNode:trailer:. This is a bit more involved, and has a bad side effect that the source code is now not in phase with generated code, which might become a problem once you want to debug the method (fortunately, Squeak can used decompiled code in place of source code!).
Last, you can also arrange to have an alternate compiler and parser for some of your classes (see compilerClass and/or parserClass). The alternate TypeHintParser would accept modified syntax with the type hints in source code (once upon a time, it was implemented with type hints following the args inside angle brackets foo: x <Integer> bar: y <Number>). And the alternate TypeHintCompiler would arrange to compile preconditions automatically given those type hints. Since you will then be very advanced in Squeak, you will also create special mapping between source code index and bytecodes so as to have sane debugger and even special Decompiler class that could recognize the precondition type checks and transform them back to type hints just in case.
My advice would be to start with the first approach that you are proposing.
EDIT
I forgot to say, there is yet another way, but it is currently available in Pharo rather than Squeak: Pharo compiler (named OpalCompiler) does reify the bytecode instructions as objects (class names beginning with IR) in the generation phase. So it is also possible to directly manipulate the bytecode instructions by proper hacking at this stage... I'm pretty sure that we can find examples of usage. Probably the most advanced technic.

Groovy optional parentheses and dots

I am learning Groovy and am pretty impressed with how it allows one to build a intelligent DSL, but I am a bit confused by the rules for when parentheses and dots are optional. Consider the following code:
Integer take(Integer x) {x}
take 3 plus 4
This works as expected and produces an output of 7 (when ran in the console), as groovy understands that last line as take(3).plus(4).
Now, println take 3 plus 4 does not work as groovy understands that as println(take).3(plus).4 which is nonsense.
Every example that I am seeing shows these sort of expression by themselves on a line, but apparently
s = take 3 plus 4
works and stores the result 7 in s. My question is, why does
println( take 3 plus 4 )
not work? Obviously, groovy will parse these sort of expressions even if they are not by themselves on a line (as shown by the assignment working). I would have thought that adding those parentheses would remove the ambiguity from the form of that line that doesn't work and that it would print out 7 as I intended.
Instead groovy gives an error 'unexpected token: 3'. As far as I can tell, groovy will not allow optional parentheses or dots inside that println, even though it doesn't seem to be ambiguous. When exactly does this sort of trick work?
This falls into the category of a nested method call, and in that case you cannot omit the parentheses. This is causing ambiguity and the results are unexpected, as the println method thinks you are passing it multiple parameters. You could reduce the ambiguity by using a groovy string in the println method.
println "${take 3 plus 4}"
More info: Omit Parentheses

Resources