I have a Groovy application that can be custimized by a small Groovy DSL I wrote. On startup, the application loads several Groovy scripts, applies some AST transformations and finally executes whatever was specified in the scripts.
One of the AST transformations inserts a couple of lines of code into certain methods. That works fine and I can see the different behavior during runtime. However, sometimes the generated code is not correct. Although I load the scripts with the TypeChecked customizer in place, my generated code is never checked for soundness.
To show my problem, I constructed an extreme example. I have the following script:
int test = 10
println test // prints 10 when executed without AST
I load this script and insert a new line of code between the declaration of test and println:
public void visitBlockStatement(BlockStatement block) {
def assignment = (new AstBuilder().buildFromSpec {
binary {
variable "test"
token "="
constant 15
}
}).first()
def newStmt = new ExpressionStatement(assignment)
newStmt.setSourcePosition(block.statements[1])
block.statements.add(2, newStmt)
super.visitBlockStatement(block)
}
After applying this AST, the script prints 15. When I use AstNodeToScriptVisitor to print the Groovy code of the resulting script, I can see the new assignment added to the code.
However, if I change the value of the assignment to a String value:
// ...
def assignment = (new AstBuilder().buildFromSpec {
binary {
variable "test"
token "="
constant "some value"
}
}).first()
// ...
I get a GroovyCastExcpetion at runtime. Although the resulting script looks like this:
int test = 10
test = "some value" // no compile error but a GroovyCastException at runtime here. WHY?
println test
no error is raised by TypeChecked. I read in this mailing list, that you need to set the source position for generated code to be checked, but I'm doing that an it still doesn't work. Can anyone provide some feedback of what I am doing wrong? Thank you very much!
Update
I call the AST by attaching it to the GroovyShell like this:
def config = new CompilerConfiguration()
config.addCompilationCustomizers(
new ASTTransformationCustomizer(TypeChecked)
)
config.addCompilationCustomizers(
new ASTTransformationCustomizer(new AddAssignmentAST())
)
def shell = new GroovyShell(config)
shell.evaluate(new File("./path/to/file.groovy"))
The class for the AST itself looks like this:
#GroovyASTTransformation(phase = CompilePhase.CANONICALIZATION)
class AddAssignmentAST implements ASTTransformation {
#Override
public void visit(ASTNode[] nodes, SourceUnit source) {
def transformer = new AddAssignmentTransformer()
source.getAST().getStatementBlock().visit(transformer)
}
private class AddAssignmentTransformer extends CodeVisitorSupport {
#Override
public void visitBlockStatement(BlockStatement block) {
// already posted above
}
}
}
Since my Groovy script only consists of one block (for this small example) the visitBlockStatement method is called exactly once, adds the assignment (which I can verify since the output changes) but does not ever throw a compile-time error.
I have a DSL where, if present, a closure called before will be called before every command.
In my setup I have 3 files: The script itself - Script, a ScriptBase, that is 'attached' to the script via a CompilerConfiguration, and a Handler.
In the script I may or may not have a closure called before.
before = {
//Do stuff.
}
Notice the lack of a type declaration, or def. If I understand Groovy correctly, this means that before is a in the binding, and accessible from outside code when evaluated with GroovyShell.evaluate().
In the ScriptBase I do the following:
class ProductSpecificationBase extends Script {
def before = null
}
This script base may or may not be overridden later on.
Then, in the Handler, I'm doing a check for whether a before closure is defined in the script:
def config = new CompilerConfiguration()
config.setScriptBaseClass(ScriptBase.class.name)
def shell = GroovyShell()
evaluatedScript = shell.evaluate(new File(thePathToScript))
if (evaluatedScript.before) {
theEvaluationOfMyScript.before()
}
The code works as expected if the script does contain a before closure, but if it doesn't it returns a MissingPropertyException. I've had a look at what this means, and it seems that my before in the ScriptBase isn't considered a property, and all the examples of using these ScriptBases I've found on the internet give examples of using methods. This is not feasible for my use case I'm afraid. How can I ensure that the closure in the ScriptBase is considered a property instead of a field(as I am assuming it is now).
To be paraphrase: I would like my code to not execute the if block if the script does not contain a before closure as well as not having been overridden in an extension of the ScriptBase. However, I would like the evaluation of evaluatedScript.before to be false as it is an empty/null Closure (i.e. it went all the way up to ScriptBase, and found the null closure)
I like to avoid a try/catch approach if possible.
in your example you would basically call the getter for the before property. To check, if there is a method with the name (and params) check with respondsTo. To see, if there is a property at all with that name use hasProperty (Thanks #dmahapatro for pointing this out)
class X {
void before() { println 'x' }
}
class Y { }
class Z {
def before = { println 'z' }
}
def x = new X()
def y = new Y()
def z = new Z()
assert x.respondsTo('before', null)
assert !y.respondsTo('before', null)
assert !z.respondsTo('before', null)
assert !x.hasProperty('before')
assert !y.hasProperty('before')
assert z.hasProperty('before')
x.before()
z.before()
My project takes in a version number (separated by '.' or '_'). I tried writing a Groovy script that creates a Jenkins environment variable using only the first two of these numbers:
//Get the version parameter
def env = System.getenv()
def version = env['currentversion']
def m = version =~/\d{1,2}/
env = ['miniVersion':m[0].m[1]]
Am I doing this correctly? Can I even create a new environment variable? Is there a better solution to this?
Jenkins 1.x
The following groovy snippet should pass the version (as you've already supplied), and store it in the job's variables as 'miniVersion'.
import hudson.model.*
def env = System.getenv()
def version = env['currentversion']
def m = version =~/\d{1,2}/
def minVerVal = m[0]+"."+m[1]
def pa = new ParametersAction([
new StringParameterValue("miniVersion", minVerVal)
])
// add variable to current job
Thread.currentThread().executable.addAction(pa)
The variable will then be accessible from other build steps. e.g.
echo miniVersion=%miniVersion%
Outputs:
miniVersion=12.34
I believe you'll need to use the "System Groovy Script" (on the Master node only) as opposed to the "Groovy Plugin" - https://wiki.jenkins-ci.org/display/JENKINS/Groovy+plugin#Groovyplugin-GroovyScriptvsSystemGroovyScript
Jenkins 2.x
I believe the previous (Jenkins 1.x) behaviour stopped working because of this Security Advisory...
Solution (paraphrased from the Security Advisory)
It's possible to restore the previous behaviour by setting the system property hudson.model.ParametersAction.keepUndefinedParameters to true. This is potentially very unsafe and intended as a short-term workaround only.
java -Dhudson.model.ParametersAction.keepUndefinedParameters=true -jar jenkins.war
To allow specific, known safe parameter names to be passed to builds, set the system property hudson.model.ParametersAction.safeParameters to a comma-separated list of safe parameter names.
e.g.
java -Dhudson.model.ParametersAction.safeParameters=miniVersion,FOO,BAR -jar jenkins.war
And in groovy these two lines should be written this way:
System.setProperty("hudson.model.ParametersAction.keepUndefinedParameters","true");
System.setProperty("hudson.model.ParametersAction.safeParameters","miniVersion,FOO,BAR");
You can also define a variable without the EnvInject Plugin within your Groovy System Script:
import hudson.model.*
def build = Thread.currentThread().executable
def pa = new ParametersAction([
new StringParameterValue("FOO", "BAR")
])
build.addAction(pa)
Then you can access this variable in the next build step which (for example) is an windows batch command:
#echo off
Setlocal EnableDelayedExpansion
echo FOO=!FOO!
This echo will show you "FOO=BAR".
Regards
For me, the following also worked in Jenkins 2 (2.73.3)
Replace
def pa = new ParametersAction([new StringParameterValue("FOO", foo)])
build.addAction(pa)
with
def pa = new ParametersAction([new StringParameterValue("FOO", foo)], ["FOO"])
build.addAction(pa)
ParametersAction seems to have a second constructor which allows to pass in "additionalSafeParameters" https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/model/ParametersAction.java
As other answers state setting new ParametersAction is the way to inject one or more environment variables, but when a job is already parameterised adding new action won't take effect. Instead you'll see two links to a build parameters pointing to the same set of parameters and the one you wanted to add will be null.
Here is a snippet updating the parameters list in both cases (a parametrised and non-parametrised job):
import hudson.model.*
def build = Thread.currentThread().executable
def env = System.getenv()
def version = env['currentversion']
def m = version =~/\d{1,2}/
def minVerVal = m[0]+"."+m[1]
def newParams = null
def pl = new ArrayList<StringParameterValue>()
pl.add(new StringParameterValue('miniVersion', miniVerVal))
def oldParams = build.getAction(ParametersAction.class)
if(oldParams != null) {
newParams = oldParams.createUpdated(pl)
build.actions.remove(oldParams)
} else {
newParams = new ParametersAction(pl)
}
build.addAction(newParams)
The Jenkins EnvInject Plugin might be able to help you. It allows injecting environment variables into the build environment.
I know it has some ability to do scripting, so it might be able to do what you want. I have only used it to set simple properties (e.g. "LOG_PATH=${WORKSPACE}\logs").
After searching around a bit, the best solution in my opinion makes use of hudson.model.EnvironmentContributingAction.
import hudson.model.EnvironmentContributingAction
import hudson.model.AbstractBuild
import hudson.EnvVars
class BuildVariableInjector {
def build
def out
def BuildVariableInjector(build, out) {
this.build = build
this.out = out
}
def addBuildEnvironmentVariable(key, value) {
def action = new VariableInjectionAction(key, value)
build.addAction(action)
//Must call this for action to be added
build.getEnvironment()
}
class VariableInjectionAction implements EnvironmentContributingAction {
private String key
private String value
public VariableInjectionAction(String key, String value) {
this.key = key
this.value = value
}
public void buildEnvVars(AbstractBuild build, EnvVars envVars) {
if (envVars != null && key != null && value != null) {
envVars.put(key, value);
}
}
public String getDisplayName() {
return "VariableInjectionAction";
}
public String getIconFileName() {
return null;
}
public String getUrlName() {
return null;
}
}
}
I use this class in a system groovy script (using the groovy plugin) within a job.
import hudson.model.*
import java.io.File;
import jenkins.model.Jenkins;
def jenkinsRootDir = build.getEnvVars()["JENKINS_HOME"];
def parent = getClass().getClassLoader()
def loader = new GroovyClassLoader(parent)
def buildVariableInjector = loader.parseClass(new File(jenkinsRootDir + "/userContent/GroovyScripts/BuildVariableInjector.groovy")).newInstance(build, getBinding().out)
def projectBranchDependencies = []
//Some logic to set projectBranchDependencies variable
buildVariableInjector.addBuildEnvironmentVariable("projectBranchDependencies", projectBranchDependencies.join(","));
You can then access the projectBranchDependencies variable at any other point in your build, in my case, from an ANT script.
Note: I borrowed / modified the ideas for parts of this implementation from a blog post, but at the time of this posting I was unable to locate the original source in order to give due credit.
Just had the same issue. Wanted to dynamically trigger parametrized downstream jobs based on the outcome of some groovy scripting.
Unfortunately on our Jenkins it's not possible to run System Groovy scripts. Therefore I had to do a small workaround:
Run groovy script which creates a properties file where the environment variable to be set is specified
def props = new File("properties.text")
if (props.text == 'foo=bar') {
props.text = 'foo=baz'
} else {
props.text = 'foo=bar'
}
Use env inject plugin to inject the variable written into this script
Inject environment variable
Property file path: properties.text
After that I was able to use the variable 'foo' as parameter for the parametrized trigger plugin. Some kind of workaround. But works!
My environment was prior tooling such as Jenkins and was running with batch files (I know, I'm old). So those batch files (and their sub-batch files) are using environment variables. This was my piece of groovy script which injects environment variables. The names and parameters used are dummy ones.
// The process/batch which uses environment variables
def buildLabel = "SomeVersionNr"
def script = "startBuild.bat"
def processBuilder = new ProcessBuilder(script, buildLabel)
//Inject our environment variables
Map<String, String> env = processBuilder.environment()
env.put("ProjectRoot", "someLocation")
env.put("SomeVar", "Some")
Process p = processBuilder.start()
p.waitFor()
Of course if you set Jenkins up from scratch you would probably do it differently and share the variables in another way, or pass parameters around but this might come handy.
On my side it only worked this way by replacing an existing parameter.
def artifactNameParam = new StringParameterValue('CopyProjectArtifactName', 'bla bla bla')
build.replaceAction(new ParametersAction(artifactNameParam))
Additionally this script must be run with system groovy.
A groovy must be manually installed on that system and the bin dir of groovy must be added to path. Additionally in the lib folder I had to add jenkins-core.jar.
Then it was possible to modify a parameter in a groovy script and get the modified value in a batch script after to continue work.
For me the following worked on Jenkins 2.190.1 and was much simpler than some of the other workarounds:
matcher = manager.getLogMatcher('^.*Text we want comes next: (.*)$');
if (matcher.matches()) {
def myVar = matcher.group(1);
def envVar = new EnvVars([MY_ENV_VAR: myVar]);
def newEnv = Environment.create(envVar);
manager.build.environments.add(0, newEnv);
// now the matched text from the LogMatcher is passed to an
// env var we can access at $MY_ENV_VAR in post build steps
}
This was using the Groovy Script plugin with no additional changes to Jenkins.
You can also create a global environment variable for jenkins if want to use wider.
Written here longer.
import hudson.EnvVars;
import hudson.slaves.EnvironmentVariablesNodeProperty;
import hudson.slaves.NodeProperty;
import hudson.slaves.NodePropertyDescriptor;
import hudson.util.DescribableList;
import jenkins.model.Jenkins;
public createGlobalEnvironmentVariables(String key, String value){
Jenkins instance = Jenkins.getInstance();
DescribableList<NodeProperty<?>, NodePropertyDescriptor> globalNodeProperties = instance.getGlobalNodeProperties();
List<EnvironmentVariablesNodeProperty> envVarsNodePropertyList = globalNodeProperties.getAll(EnvironmentVariablesNodeProperty.class);
EnvironmentVariablesNodeProperty newEnvVarsNodeProperty = null;
EnvVars envVars = null;
if ( envVarsNodePropertyList == null || envVarsNodePropertyList.size() == 0 ) {
newEnvVarsNodeProperty = new hudson.slaves.EnvironmentVariablesNodeProperty();
globalNodeProperties.add(newEnvVarsNodeProperty);
envVars = newEnvVarsNodeProperty.getEnvVars();
} else {
envVars = envVarsNodePropertyList.get(0).getEnvVars();
}
envVars.put(key, value)
instance.save()
}
createGlobalEnvironmentVariables('Var1','Dummy')
A SoapUI project can run random script upon load.
Load Script is invoked with log and project variables.
In my shared lib I have method - addAsserts() that traverses the whole project and adds schema compliance assertions to SOAP test steps. In my Load Script I call shared method
addAsserts(this)
passing 'this' as a parameter and set closure.delegate to it inside addAsserts method to make 'project' variable accessible within the closure scope
addAsserts method is defined in sharedUtil.groovy:
static def addAsserts(that){
def closure={
project.testSuites.each { testSuiteName, testSuiteObject ->
testSuiteObject.testCases.each { testCaseName, testCaseObject ->
testCaseObject.testSteps.each { testStepName, testStepObject ->
if ("class com.eviware.soapui.impl.wsdl.teststeps.WsdlTestRequestStep" == testStepObject.getClass().toString() ) {
log.info "adding 'Schema Compliance' assertion to ${testSuiteName}/${testCaseName}/${testStepName}"
testStepObject.addAssertion('Schema Compliance')
}
}
}
}
}//closure
closure.delegate=that // <--- i would like NOT to pass 'that' as parameter
// but rather detect in runtime with some kind of
// getCallerInstance() method
return closure.call()
}
QUESTION:
Is it possible to detect caller instance in runtime with some kind of getCallerInstance() method ?
No, I don't believe this is possible. Wasn't in Java either (you can find out the name/method of the calling class using some horrible stacktrace hacking, but not the instance of the class itself)
Edit...
It might be possible with a Category (but I am not experienced with SoapUI, so I don't know if this technique would fit)
Say we have a class Example defined like so:
class Example {
String name
}
We can then write a class very similar to your example code, which in this case will set the delegate of the closure, and the closure will print out the name property of the delegate (as we have set the resolve strategy to DELEGATE_ONLY)
class AssetAddingCategory {
static def addAsserts( that ) {
def closure = {
"Name of object: $name"
}
closure.delegate = that
closure.resolveStrategy = Closure.DELEGATE_ONLY
closure.call()
}
}
Later on in our code, it is then possible to do:
def tim = new Example( name:'tim' )
use( AssetAddingCategory ) {
println tim.addAsserts()
}
And this will print out
Name of object: tim
The output from println from within a class function is lost.
An example script (outputclass.groovy):
class OutputClass
{
OutputClass()
{
println("Inside class") // This will not show in the console
}
}
println("Outside class") // Only this is shown in the console
output = new OutputClass()
I use Jenkins CLI to execute the groovy script
java -jar ..\jenkins-cli.jar -s JENKINS_SERVER_URL groovy outputclass.groovy
It only outputs this:
Outside class
It seems like the class inmplicitly uses println from System.out.println, and System.out is directed to the log files, but the println outside the class is using something else, which is outputted in the script console. The following code shows the behavior.
System.out.println("First")
println("Second")
Output:
Second
How do I explicitly set the output device to output to the Jenkins script console?
I found the solution myself here http://mriet.wordpress.com.
When the Groovy plugin starts is passes two bindings to the script. From the bindings we can get the out variable. Get it and use out.println to output to the script console, not the plain println.
The script below shows full solution.
import hudson.model.*
// Get the out variable
def out = getBinding().out;
class OutputClass
{
OutputClass(out) // Have to pass the out variable to the class
{
out.println ("Inside class")
}
}
out.println("Outside class")
output = new OutputClass(out)
If you use the skript as a post build step (I'm not shure whether it works with the mentioned CLI) you can use the build in logger:
manager.listener.logger.println("some output")
So in your case something like this may be helpful:
class OutputClass
{
OutputClass(logger) // Have to pass the out variable to the class
{
logger.println ("Inside class")
}
}
output = new OutputClass(manager.listener.logger)
See also Example 10 in Groovy Plugin Doc
Does this mailing list post help?
the output is sent to standard output, so if you check your log file, you
will probably see something like this: INFO [STDOUT] Hello World
if you insist on using system script, you have to pass out variable to
your class, as the binding is not visible inside the class (so it's
passed to standard output). You should use something like this
public class Hello {
static void say(out) {
out << "Hello World "
}
}
println "Started ..."
Hello.say(out)
A simple solution that worked well for me was to add this line on top of each script. This enables usage of traditional println commands all over the code (inside and outside of classes) leaving the code intuitive.
import hudson.model.*
System.out = getBinding().out;
This enables to create log entries like this:
println("Outside class");
class OutputClass {
OutputClass() {
println ("Inside class")
}
}
new OutputClass();
It replaces the default print stream in System.out with the one handed over from Jenkins via bindings.