There is a groovy script that has a function defined and used in multiple threads.
I found that time to time it mixes some variable values with other threads.
The problem appears when developer forgot to declare variable like this:
def f ( x ) {
y = "" + x
println y
}
The problem disappears when developer declares variable
def f ( x ) {
def y = "" + x
println y
}
In classes there is no way to use undefined variables.
The reason is that in the scripts the undefined variable acts as an instance variable of the script-class. Actually this is a binding for external variables that could be passed into the script.
Here is a part of script that demonstrates the problem of using undefined variables in several threads.
void f(String x){
y=""+x; //if you put def at this line it'll work fine
Thread.sleep(333);
//usually developers expected that `y` is a local variable,
//but without declaration it belongs to script-class
if( !y.equals(x) ) println("failure: x=$x y=$y");
}
//thead 1 start
Thread.start{
for(int i=0;i<20;i++){
f( i.toString() )
Thread.sleep(100);
}
}
//thead 2 start
Thread.start{
for(int i=0;i<20;i++){
f( i.toString() )
Thread.sleep(150);
}
}
//main thread sleep.
Thread.sleep(2000);
println("done");
this code will print out failures when x not equals y (literally)
Write a compiler configuration using a scriptBaseClass to disallow undeclared variables and the usage of the script's own binding.
This is the base script (my DefBase.groovy file):
abstract class NoUndefShallPass extends Script {
void setProperty(String name, val) {
// seems like groovy itself set 'args' in the binding, probably from CL
assert name == 'args',
"Error in '$name'; variables should be declared using 'def'"
}
}
def configuration = new org.codehaus.groovy.control.CompilerConfiguration()
configuration.setScriptBaseClass(NoUndefShallPass.class.name)
def shell = new GroovyShell(this.class.classLoader, new Binding(), configuration)
shell.evaluate new File('/tmp/Defing.groovy')
And the script. It will thrown an AssertionError if the setProperty tries to use the binding:
void f(String x){
y=""+x; //if you put def at this line it'll work fine
Thread.sleep(333);
if( !y.equals(x) ) println("failure: x=$x y=$y");
}
def t1 = Thread.start{
20.times { i ->
f( i.toString() )
Thread.sleep(100);
}
}
def t2 = Thread.start{
20.times { i ->
f( i.toString() )
Thread.sleep(150);
}
}
Thread.sleep(2000);
t1.join()
t2.join()
println("done");
it's poorly described here as "the binding"
http://groovy.codehaus.org/Scoping+and+the+Semantics+of+%22def%22
Related
Is there a way to skip MissingPropertyException while using GroovyShell.evaluate?
def sharedData = new Binding()
def shell = new GroovyShell(sharedData)
shell.evaluate("a=5; b=1") // works fine
// How to not get MissingPropertyException, or silently ignore property 'a'
shell.evaluate("a; b=1") // MissingPropertyException for 'a'
I know about the Expando solution, is there a way to do it without defining a class?
A very minimal approach would be to override Binding.getVariable.
Note that this is very straight forward: "all exceptions" are ignored - you might want to have better logging or more accurate error handling.
import groovy.lang.*
class NoOpBinding extends Binding {
#Override
Object getVariable(String name) {
try {
return super.getVariable(name)
}
catch (Throwable t) {
println "Ignoring variable=`$name`"
return null
}
}
}
def shell = new GroovyShell(new NoOpBinding())
shell.evaluate("a; b=1") // MissingPropertyException for 'a'
// → Ignoring variable=`a`
println shell.getVariable('b')
// → 1
You could do something like this:
def binding = [:].withDefault { }
def shell = new GroovyShell(binding as Binding)
shell.evaluate 'a; b=1'
I have these 3 groovy files:
A.groovy:
// ...
stage("Test")
{
throw new Exception("This script fails")
}
B.groovy:
// ...
stage("Test")
{
// Nothing, want this to pass
}
main.groovy:
// ...
m = [:]
status = [:]
scripts = ["A", "B"]
for script in scripts
{
m["${script}"] =
{
stage("${script}")
{
try
{
load "${script}.groovy"
status["${script}"] = true
}
catch (Exception e)
{
status["${script}"] = false
}
}
}
}
parallel m
for result_iterator in status:
print "${result_iterator.key} resulted in ${result_iterator.value}"
The code above is a sketch of the real code =)
When I run main.groovy to see the results in the status dictionary, I only get to see B. A threw an exception and thus it didn't add itself to the dictionary. Is there a way I can catch A's exception somehow?
Problem 1
You got bitten by the way Groovy closures capture variables outside of their scope. The references to script inside the closure will receive the value of script at the time when the closure is run, which will be the last value of script. So both closures actually load "B.groovy" and the exception is never thrown!
The fix is simple, just create a local variable inside of the loop that captures the current value of script:
for( script in scripts ) {
def theScript = script
// Now only use theScript inside of the closure!
}
Problem 2
You are not serializing write access to the status variable from parallel threads. This could result in very nasty, hard-to-reproduce bugs.
The fix is to use Map.asSynchronized() to create a synchronized map:
status = [:].asSynchronized()
Complete 'main.groovy' fix
m = [:]
status = [:].asSynchronized() // because multiple threads write to this variable
scripts = ["A", "B"]
for( script in scripts ) {
def theScript = script // Important to pass the current value into the closure!
m[ theScript ] = {
stage( theScript ) {
try {
load "${theScript}.groovy"
status[ theScript ] = true
}
catch (Exception e) {
status[ theScript ] = false
}
}
}
}
parallel m
for( result_iterator in status ) {
print "${result_iterator.key} resulted in ${result_iterator.value}"
}
Note: I have also removed the unnecessary string interpolations.
I'm using Groovy and Gradle for my testing.
I have the following lines in my app code:
Platform.runLater( new Runnable() {
void run() {
....
FileChooser fc = getFileChooser()
File file = fc.showOpenDialog( null )
// if( file != null && file.exists() ) { <--- this is what I have to put
if( file.exists() ) {
println( "file $file exists")
analyseFile( file )
}
If I mock (using GroovyMock because javafx.stage.FileChooser is final) so that fc.showOpenDialog returns null I would expect a NullPointerException to be thrown on file.exists()... and one is.
But this doesn't show up in the test results, which are all green. The only way you find that this has happened is when you go and look at the test results for the class: then you see that the grey button saying "StdErr" is present.
This appears to be because the enveloping Runnable is "swallowing" it...
Is there any way in Spock to make an Exception inside a Runnable lead to test failure?
PS test code:
def "getting a null result from the showOpenDialog should be handled OK"(){
given:
FileChooser fc = GroovyMock( FileChooser )
ConsoleHandler ch = Spy( ConsoleHandler ){ getFileChooser() >> fc }
ch.setMaxLoopCount 10
systemInMock.provideLines( "o" )
fc.showOpenDialog( _ ) >> null
when:
com.sun.javafx.application.PlatformImpl.startup( {} )
ch.loop()
Thread.sleep( 1000L ) // NB I know this is a rubbish way to handle things... but I'm a newb with Spock!
then:
0 * ch.analyseFile( _ )
}
here's the SSCCE for the app code
class ConsoleHandler {
int loopCount = 0
def maxLoopCount = Integer.MAX_VALUE - 1
def response
FileChooser fileChooser = new FileChooser()
void analyseFile( File file ) {
// ...
}
void loop() {
while( ! endConditionMet() ){
print '> '
response = System.in.newReader().readLine()
if( response ) response = response.toLowerCase()
if( response == 'h' || response == 'help' ){
println "Help: enter h for this help, q to quit"
}
else if ( response == 'o' ){
Platform.runLater( new Runnable() {
void run() {
FileChooser fc = getFileChooser()
File file = fc.showOpenDialog( null )
// if( file != null && file.exists() ) {
if( file.exists() ) {
analyseFile( file )
}
}
})
}
}
}
boolean endConditionMet() {
loopCount++
response == 'q' || loopCount > maxLoopCount
}
static void main( args ) {
com.sun.javafx.application.PlatformImpl.startup( {} )
new ConsoleHandler().loop()
com.sun.javafx.application.PlatformImpl.exit()
}
}
The only other thing that might require an explanation is systemInMock in the Spock test code, which lets the test supply text to StdIn in the app code:
import org.junit.contrib.java.lang.system.TextFromStandardInputStream
import static org.junit.contrib.java.lang.system.TextFromStandardInputStream.emptyStandardInputStream
....
// field of the Test class, of course:
#Rule
public TextFromStandardInputStream systemInMock = emptyStandardInputStream()
The corresponding line in my Gradle dependencies clause is :
testCompile 'com.github.stefanbirkner:system-rules:1.16.0'
... but I think this could be obtained by using a #Grab in the Groovy test file, right?
I think I now realise this question is really about how best to do JavaFX-app-thread testing in Spock... EDT testing with Swing was always a problem, no doubt similar things apply. And it's all coming back to me: when you get Platform to run the Runnable it would make no sense to throw an Exception from runLater as the caller code has moved on, so I believe the only realistic option is for run() to call some other method which the testing code can call...
I have searched for "Spock JavaFX testing" but not much has come up...
This is what I imagine to be the only practical way forward: change this bit like this:
else if ( response == 'o' ){
Platform.runLater(new Runnable() {
void run() {
openFileChooser()
}
})
}
new method for ConsoleHandler:
def openFileChooser() {
FileChooser fc = getFileChooser()
File file = fc.showOpenDialog( null )
// if( file != null && file.exists() ) {
if( file.exists() ) {
analyseFile( file )
}
}
new Spock test:
def "getting a null result from the showOpenDialog should be handled OK2"(){
given:
FileChooser fc = GroovyMock( FileChooser )
ConsoleHandler ch = Spy( ConsoleHandler ){ getFileChooser() >> fc }
fc.showOpenDialog( _ ) >> null
when:
com.sun.javafx.application.PlatformImpl.startup( {} )
ch.openFileChooser()
Thread.sleep( 1000L )
then:
0 * ch.analyseFile( _ )
}
Test fails (hurrah!):
java.lang.NullPointerException: Cannot invoke method exists() on null
object
But I'd welcome input about whether this can be improved, from more experienced Groovy/Spock hands.
Indeed I was slight puzzled why I didn't get "incorrect State" when I called openFileChooser in the when: clause. I presume this is because going showOpenDialog on a mock FileChooser doesn't actually involve any check to make sure we're currently in the JavaFX-app-thread. Turns out the ...PlatformImpl.startup( {} ) and Thread.sleep lines can be deleted in that test. Is this ideal practice though?! Hmmm...
later
Closing this as the only answer... in reality the same sorts of considerations apply to this question as to testing code in plain-old Java using plain-old JUnit in the JavaFX-app-thread. Spock + Groovy may pose some problems or they may make it easier... I am not yet experienced enough with them to know.
In Groovy, I can overload operator '+' plus as follow:
class MutableInt {
int val
MutableInt(int val) { this.val = val }
MutableInt plus(int val) {
return new MutableInt(this.val += val)
}
}
The above class works fine for the following test cases:
def m1 = new MutableInt(1);
assert (m1 + 1).val == 2;
However, if I need to use it together with Map like this and compile it with static
#groovy.transform.CompileStatic
void compileItWithStatic() {
Map<Long, MutableInt> mutMap = [:].withDefault{ new MutableInt(0) }
assert (mutMap[1L] += 20).val == 20;
}
compileItWithStatic()
I got the following error:
*Script1.groovy: 17: [Static type checking] -
Cannot call <K,V> java.util.Map <java.lang.Long, MutableInt>#putAt(java.lang.Long, MutableInt) with arguments [long, int]*
How can I override the '+=' operator and compile it with static without error?
EDIT:
If I am doing like this without compile static it works fine:
def m1 = new MutableInt(1);
assert (m1 += 1).val == 2 // <----- caution: '+=' not '+' as in previous case
However, if it was inside the method like this:
#groovy.transform.CompileStatic
void compileItWithStatic_2() {
def m1 = new MutableInt(1);
assert (m1 += 1).val == 2
}
The error will be:
Script1.groovy: -1: Access to java.lang.Object#val is forbidden # line -1, column -1.
1 error
P.S. It won't work with static compilation not with dynamic compilation.
The assignment part is throwing the error. A simple + works:
class MutableInt {
int val
MutableInt(int val) { this.val = val }
MutableInt plus(int val) {
return new MutableInt(this.val += val)
}
}
def m1 = new MutableInt(1);
assert (m1 + 1).val == 2;
#groovy.transform.CompileStatic
def compileItWithStatic() {
Map<Long, MutableInt> mutMap = [:].withDefault{ new MutableInt(0) }
mutMap[1L] + 20
mutMap
}
assert compileItWithStatic()[1L].val == 20
Groovy is parsing mutMap[1L] += 20 as mutMap.putAt(1L, 20). This looks like a bug to me. This works: mutMap[1L] = mutMap[1L] + 20, albeit more verbose.
Edit: the second error seems related to the result of the expression (m1 + 1) being parsed as Object. This should work:
#groovy.transform.CompileStatic
void compileItWithStatic_2() {
def m1 = new MutableInt(1) + 1;
assert m1.val == 2
}
Let's say I have a DSL like this
setup {name = "aDSLScript"}
println "this is common groovy code"
doStuff {println "I'm doing dsl stuff"}
One would have a delegating class implementing the methods 'setup' and 'doStuff' usually. Beside, one could write common Groovy code to be executed (println...).
What I am searching for, is a way to execute this in two steps. In the first step only the setup method should be processed (neither println). The second step handles the other parts.
At the moment, I have two delegating classes. One implements 'setup' the other one implements 'doStuff'. But both execute the println statement, of course.
You can create a single class to intercept the method calls from the script and let it coordinate the following method invoke. I did it through reflection, but you can go declarative if you want. These are the model and script classes:
class FirstDelegate {
def setup(closure) { "firstDelegate.setup" }
}
class SecondDelegate {
def doStuff(closure) { "secondDelegate.doStuff" }
}
class MethodInterceptor {
def invokedMethods = []
def methodMissing(String method, args) {
invokedMethods << [method: method, args: args]
}
def delegate() {
def lookupCalls = { instance ->
def invokes = instance.metaClass.methods.findResults { method ->
invokedMethods.findResult { invocation ->
invocation.method == method.name ?
[method: method, invocation: invocation] : null
}
}
invokes.collect { invoked ->
invoked.method.invoke(instance, invoked.invocation.args)
}
}
return lookupCalls(new FirstDelegate()) + lookupCalls(new SecondDelegate())
}
}
Here be scripts and assertions:
import org.codehaus.groovy.control.CompilerConfiguration
def dsl = '''
setup {name = "aDSLScript"}
println "this is common groovy code"
doStuff {println "Ima doing dsl stuff"}
'''
def compiler = new CompilerConfiguration()
compiler.scriptBaseClass = DelegatingScript.class.name
def shell = new GroovyShell(this.class.classLoader, new Binding(), compiler)
script = shell.parse dsl
interceptor = new MethodInterceptor()
script.setDelegate interceptor
script.run()
assert interceptor.invokedMethods*.method == [ 'setup', 'doStuff' ]
assert interceptor.delegate() ==
['firstDelegate.setup', 'secondDelegate.doStuff']
Notice I didn't bothered intercepting println call, which is a DefaultGroovyMethods thus, a little more cumbersome to handle.
Also having the class MethodInterceptor implementing the method delegate() is not a good idea, since this allows the user-defined script to call it.
I found a way to split up execution of the DSL script. I used a CompilationCustomizer to remove every statement from AST except the doFirst{}. So the first run will only execute doFirst. The second run does everything else. Here's some code:
class DoFirstProcessor {
def doFirst(Closure c) {
c()
}
}
class TheRestProcessor {
def doStuff(Closure c) {
c()
}
def methodMissing(String name, args) {
//nothing to do
}
}
def dsl = "
println 'this is text that will not be printed out in first line!'
doFirst { println 'First things first: e.g. setting up environment' }
doStuff { println 'doing some stuff now' }
println 'That is it!'
"
class HighlanderCustomizer extends CompilationCustomizer {
def methodName
HighlanderCustomizer(def methodName) {
super(CompilePhase.SEMANTIC_ANALYSIS)
this.methodName = methodName
}
#Override
void call(SourceUnit sourceUnit, GeneratorContext generatorContext, ClassNode classNode) throws CompilationFailedException {
def methods = classNode.getMethods()
methods.each { MethodNode m ->
m.code.each { Statement st ->
if (!(st instanceof BlockStatement)) {
return
}
def removeStmts = []
st.statements.each { Statement bst ->
if (bst instanceof ExpressionStatement) {
def ex = bst.expression
if (ex instanceof MethodCallExpression) {
if (!ex.methodAsString.equals(methodName)) {
removeStmts << bst
}
} else {
removeStmts << bst
}
} else {
removeStmts << bst
}
}
st.statements.removeAll(removeStmts)
}
}
}
}
def cc = new CompilerConfiguration()
cc.addCompilationCustomizers new HighlanderCustomizer("doFirst")
cc.scriptBaseClass = DelegatingScript.class.name
def doFirstShell = new GroovyShell(new Binding(), cc)
def doFirstScript = doFirstShell.parse dsl
doFirstScript.setDelegate new DoFirstProcessor()
doFirstScript.run()
cc.compilationCustomizers.clear()
def shell = new GroovyShell(new Binding(), cc)
def script = shell.parse dsl
script.setDelegate new TheRestProcessor()
script.run()
I did another variation of this where I execute the DSL in one step. See my blog post about it: http://hackserei.metacode.de/?p=247