Groovy - Type check AST generated code - groovy

I have a Groovy application that can be custimized by a small Groovy DSL I wrote. On startup, the application loads several Groovy scripts, applies some AST transformations and finally executes whatever was specified in the scripts.
One of the AST transformations inserts a couple of lines of code into certain methods. That works fine and I can see the different behavior during runtime. However, sometimes the generated code is not correct. Although I load the scripts with the TypeChecked customizer in place, my generated code is never checked for soundness.
To show my problem, I constructed an extreme example. I have the following script:
int test = 10
println test // prints 10 when executed without AST
I load this script and insert a new line of code between the declaration of test and println:
public void visitBlockStatement(BlockStatement block) {
def assignment = (new AstBuilder().buildFromSpec {
binary {
variable "test"
token "="
constant 15
}
}).first()
def newStmt = new ExpressionStatement(assignment)
newStmt.setSourcePosition(block.statements[1])
block.statements.add(2, newStmt)
super.visitBlockStatement(block)
}
After applying this AST, the script prints 15. When I use AstNodeToScriptVisitor to print the Groovy code of the resulting script, I can see the new assignment added to the code.
However, if I change the value of the assignment to a String value:
// ...
def assignment = (new AstBuilder().buildFromSpec {
binary {
variable "test"
token "="
constant "some value"
}
}).first()
// ...
I get a GroovyCastExcpetion at runtime. Although the resulting script looks like this:
int test = 10
test = "some value" // no compile error but a GroovyCastException at runtime here. WHY?
println test
no error is raised by TypeChecked. I read in this mailing list, that you need to set the source position for generated code to be checked, but I'm doing that an it still doesn't work. Can anyone provide some feedback of what I am doing wrong? Thank you very much!
Update
I call the AST by attaching it to the GroovyShell like this:
def config = new CompilerConfiguration()
config.addCompilationCustomizers(
new ASTTransformationCustomizer(TypeChecked)
)
config.addCompilationCustomizers(
new ASTTransformationCustomizer(new AddAssignmentAST())
)
def shell = new GroovyShell(config)
shell.evaluate(new File("./path/to/file.groovy"))
The class for the AST itself looks like this:
#GroovyASTTransformation(phase = CompilePhase.CANONICALIZATION)
class AddAssignmentAST implements ASTTransformation {
#Override
public void visit(ASTNode[] nodes, SourceUnit source) {
def transformer = new AddAssignmentTransformer()
source.getAST().getStatementBlock().visit(transformer)
}
private class AddAssignmentTransformer extends CodeVisitorSupport {
#Override
public void visitBlockStatement(BlockStatement block) {
// already posted above
}
}
}
Since my Groovy script only consists of one block (for this small example) the visitBlockStatement method is called exactly once, adds the assignment (which I can verify since the output changes) but does not ever throw a compile-time error.

Related

How to traverse AST tree

I'm trying to create an static analysis for Groovy. As a POC for my superiors I'm just trying to parse simple code and detect SQL injections, which are the easiest kind to spot. I did it successfully on Python, which is my main language, but my company mostly uses Grails (on Groovy).
This is what I have so far:
import org.codehaus.groovy.ast.expr.*;
import org.codehaus.groovy.ast.stmt.*;
import org.codehaus.groovy.ast.*
import org.codehaus.groovy.control.CompilePhase
import org.codehaus.groovy.ast.CodeVisitorSupport
import org.codehaus.groovy.ast.builder.AstBuilder
public class SecurityCheck extends CodeVisitorSupport {
void visitBlockStatement(BlockStatement statement) {
println "NEW BLOCK STATEMENT:"
println statement.getText();
//keep walking...
statement.getStatements().each { ASTNode child ->
println "CHILD FOUND: "
println child.getText();
child.visit(this)
}
}
}
def code = new File('groovy_source.groovy').text // get the code from the source file
def AstBuilder astBuilder = new AstBuilder() // build an instance of the ast builder
def ast = astBuilder.buildFromString(CompilePhase.CONVERSION, code) // build from string when the compiler converts from tokens to AST
def SecurityCheck securityCheck = new SecurityCheck() // create an instance of our security check class
println ast
println ast[0]
ast[0].visit(securityCheck)
The groovy_source.groovy file is very simple, containing only a minimal file with a super easy to spot vulnerability:
def post(id) {
query = "SELECT * FROM table WHERE id = " + id;
result = sql.execute query
return result;
}
It is my understanding that, as I'm inheriting from CodeVisitorSupport, this would just visit a BlockStatement and then, for each statement inside that statement, it would visit it using the method from the supper class.
Nevertheless, when I print the text from the BlockStatement, it is an empty string, and the for each method never even gets called (which I assume must mean the AST found no children for my block statement, even when the function definitively has statements inside it.
[org.codehaus.groovy.ast.stmt.BlockStatement#363a52f[]] // println ast
org.codehaus.groovy.ast.stmt.BlockStatement#363a52f[] // println ast[0]
NEW BLOCK STATEMENT:
{ } // println statement.getText()
Any help here would be tremendously appreciated. Thanks!
I found the answer. I wasn't so hard in the end, but the horrible documentation doesn't make it easy. If you one to traverse the tree, you need to give the constructor the false boolean as a second argument, like this:
def ast = astBuilder.buildFromString(CompilePhase.CONVERSION, false, code)
Then you can use the visit* methods as you expect.

Properties in Groovy base scripts

I have a DSL where, if present, a closure called before will be called before every command.
In my setup I have 3 files: The script itself - Script, a ScriptBase, that is 'attached' to the script via a CompilerConfiguration, and a Handler.
In the script I may or may not have a closure called before.
before = {
//Do stuff.
}
Notice the lack of a type declaration, or def. If I understand Groovy correctly, this means that before is a in the binding, and accessible from outside code when evaluated with GroovyShell.evaluate().
In the ScriptBase I do the following:
class ProductSpecificationBase extends Script {
def before = null
}
This script base may or may not be overridden later on.
Then, in the Handler, I'm doing a check for whether a before closure is defined in the script:
def config = new CompilerConfiguration()
config.setScriptBaseClass(ScriptBase.class.name)
def shell = GroovyShell()
evaluatedScript = shell.evaluate(new File(thePathToScript))
if (evaluatedScript.before) {
theEvaluationOfMyScript.before()
}
The code works as expected if the script does contain a before closure, but if it doesn't it returns a MissingPropertyException. I've had a look at what this means, and it seems that my before in the ScriptBase isn't considered a property, and all the examples of using these ScriptBases I've found on the internet give examples of using methods. This is not feasible for my use case I'm afraid. How can I ensure that the closure in the ScriptBase is considered a property instead of a field(as I am assuming it is now).
To be paraphrase: I would like my code to not execute the if block if the script does not contain a before closure as well as not having been overridden in an extension of the ScriptBase. However, I would like the evaluation of evaluatedScript.before to be false as it is an empty/null Closure (i.e. it went all the way up to ScriptBase, and found the null closure)
I like to avoid a try/catch approach if possible.
in your example you would basically call the getter for the before property. To check, if there is a method with the name (and params) check with respondsTo. To see, if there is a property at all with that name use hasProperty (Thanks #dmahapatro for pointing this out)
class X {
void before() { println 'x' }
}
class Y { }
class Z {
def before = { println 'z' }
}
def x = new X()
def y = new Y()
def z = new Z()
assert x.respondsTo('before', null)
assert !y.respondsTo('before', null)
assert !z.respondsTo('before', null)
assert !x.hasProperty('before')
assert !y.hasProperty('before')
assert z.hasProperty('before')
x.before()
z.before()

how to detect caller instance in SoapUI groovy script?

A SoapUI project can run random script upon load.
Load Script is invoked with log and project variables.
In my shared lib I have method - addAsserts() that traverses the whole project and adds schema compliance assertions to SOAP test steps. In my Load Script I call shared method
addAsserts(this)
passing 'this' as a parameter and set closure.delegate to it inside addAsserts method to make 'project' variable accessible within the closure scope
addAsserts method is defined in sharedUtil.groovy:
static def addAsserts(that){
def closure={
project.testSuites.each { testSuiteName, testSuiteObject ->
testSuiteObject.testCases.each { testCaseName, testCaseObject ->
testCaseObject.testSteps.each { testStepName, testStepObject ->
if ("class com.eviware.soapui.impl.wsdl.teststeps.WsdlTestRequestStep" == testStepObject.getClass().toString() ) {
log.info "adding 'Schema Compliance' assertion to ${testSuiteName}/${testCaseName}/${testStepName}"
testStepObject.addAssertion('Schema Compliance')
}
}
}
}
}//closure
closure.delegate=that // <--- i would like NOT to pass 'that' as parameter
// but rather detect in runtime with some kind of
// getCallerInstance() method
return closure.call()
}
QUESTION:
Is it possible to detect caller instance in runtime with some kind of getCallerInstance() method ?
No, I don't believe this is possible. Wasn't in Java either (you can find out the name/method of the calling class using some horrible stacktrace hacking, but not the instance of the class itself)
Edit...
It might be possible with a Category (but I am not experienced with SoapUI, so I don't know if this technique would fit)
Say we have a class Example defined like so:
class Example {
String name
}
We can then write a class very similar to your example code, which in this case will set the delegate of the closure, and the closure will print out the name property of the delegate (as we have set the resolve strategy to DELEGATE_ONLY)
class AssetAddingCategory {
static def addAsserts( that ) {
def closure = {
"Name of object: $name"
}
closure.delegate = that
closure.resolveStrategy = Closure.DELEGATE_ONLY
closure.call()
}
}
Later on in our code, it is then possible to do:
def tim = new Example( name:'tim' )
use( AssetAddingCategory ) {
println tim.addAsserts()
}
And this will print out
Name of object: tim

Groovy-script in jenkins println output disappears when called inside class environment

The output from println from within a class function is lost.
An example script (outputclass.groovy):
class OutputClass
{
OutputClass()
{
println("Inside class") // This will not show in the console
}
}
println("Outside class") // Only this is shown in the console
output = new OutputClass()
I use Jenkins CLI to execute the groovy script
java -jar ..\jenkins-cli.jar -s JENKINS_SERVER_URL groovy outputclass.groovy
It only outputs this:
Outside class
It seems like the class inmplicitly uses println from System.out.println, and System.out is directed to the log files, but the println outside the class is using something else, which is outputted in the script console. The following code shows the behavior.
System.out.println("First")
println("Second")
Output:
Second
How do I explicitly set the output device to output to the Jenkins script console?
I found the solution myself here http://mriet.wordpress.com.
When the Groovy plugin starts is passes two bindings to the script. From the bindings we can get the out variable. Get it and use out.println to output to the script console, not the plain println.
The script below shows full solution.
import hudson.model.*
// Get the out variable
def out = getBinding().out;
class OutputClass
{
OutputClass(out) // Have to pass the out variable to the class
{
out.println ("Inside class")
}
}
out.println("Outside class")
output = new OutputClass(out)
If you use the skript as a post build step (I'm not shure whether it works with the mentioned CLI) you can use the build in logger:
manager.listener.logger.println("some output")
So in your case something like this may be helpful:
class OutputClass
{
OutputClass(logger) // Have to pass the out variable to the class
{
logger.println ("Inside class")
}
}
output = new OutputClass(manager.listener.logger)
See also Example 10 in Groovy Plugin Doc
Does this mailing list post help?
the output is sent to standard output, so if you check your log file, you
will probably see something like this: INFO [STDOUT] Hello World
if you insist on using system script, you have to pass out variable to
your class, as the binding is not visible inside the class (so it's
passed to standard output). You should use something like this
public class Hello {
static void say(out) {
out << "Hello World "
}
}
println "Started ..."
Hello.say(out)
A simple solution that worked well for me was to add this line on top of each script. This enables usage of traditional println commands all over the code (inside and outside of classes) leaving the code intuitive.
import hudson.model.*
System.out = getBinding().out;
This enables to create log entries like this:
println("Outside class");
class OutputClass {
OutputClass() {
println ("Inside class")
}
}
new OutputClass();
It replaces the default print stream in System.out with the one handed over from Jenkins via bindings.

External Content with Groovy BuilderSupport

I've built a custom builder in Groovy by extending BuilderSupport. It works well when configured like nearly every builder code sample out there:
def builder = new MyBuilder()
builder.foo {
"Some Entry" (property1:value1, property2: value2)
}
This, of course, works perfectly. The problem is that I don't want the information I'm building to be in the code. I want to have this information in a file somewhere that is read in and built into objects by the builder. I cannot figure out how to do this.
I can't even make this work by moving the simple entry around in the code.
This works:
def textClosure = { "Some Entry" (property1:value1, property2: value2) }
builder.foo(textClosure)
because textClosure is a closure.
If I do this:
def text = '"Some Entry" (property1:value1, property2: value2)'
def textClosure = { text }
builder.foo(textClosure)
the builder only gets called for the "foo" node. I've tried many variants of this, including passing the text block directly into the builder without wrapping it in a closure. They all yield the same result.
Is there some way I take a piece of arbitrary text and pass it into my builder so that it will be able to correctly parse and build it?
Your problem is that a String is not Groovy code. The way ConfigSlurper handles this is to compile the text into an instance of Script using GroovyClassLoader#parseClass. e.g.,
// create a Binding subclass that delegates to the builder
class MyBinding extends Binding {
def builder
Object getVariable(String name) {
return { Object... args -> builder.invokeMethod(name,args) }
}
}
// parse the script and run it against the builder
new File("foo.groovy").withInputStream { input ->
Script s = new GroovyClassLoader().parseClass(input).newInstance()
s.binding = new MyBinding(builder:builder)
s.run()
}
The subclass of Binding simply returns a closure for all variables that delegates the call to the builder. So assuming foo.groovy contains:
foo {
"Some Entry" (property1:value1, property2: value2)
}
It would be equivalent to your code above.
I think the problem you described is better solved with a slurper or parser.
See:
http://groovy.codehaus.org/Reading+XML+using+Groovy%27s+XmlSlurper
http://groovy.codehaus.org/Reading+XML+using+Groovy%27s+XmlParser
for XML based examples.
In your case. Given the XML file:
<foo>
<entry name='Some Entry' property1="value1" property2="value2"/>
</foo>
You could slurp it with:
def text = new File("test.xml").text
def foo = new XmlSlurper().parseText(text)
def allEntries = foo.entry
allEntries.each {
println it.#name
println it.#property1
println it.#property2
}
Originally, I wanted to be able to specify
"Some Entry" (property1:value1, property2: value2)
in an external file. I'm specifically trying to avoid XML and XML-like syntax to make these files easier for regular users to create and modify. My current solution uses ConfigSlurper and the file now looks like:
"Some Entry"
{
property1 = value1
property2 = value2
}
ConfigSlurper gives me a map like this:
["Some Entry":[property1:value1,property2:value2]]
It's pretty simple to use these values to create my objects, especially since I can just pass the property/value map into the constructor.

Resources