On Jenkins, using Job DLS plugin, I'm trying to prepare one script which will create jobs configured for different environments (dev and preprod). Depends on for which environments this job has to run, different parameters are needed.
In this situation how to define, in the shortest way, that parameters for dev environment include the same as preprod parameters plus additionally i.e. 2 more?
An example of the code which I use is presented below.
def environments = ["DEV", "PREPROD"]
def names = ["name1", "name2", "name3"]
def jobParameters = {
string {
name("browser")
defaultValue("CHROME")
description("Browser on which one tests will run")
trim(true)
}
string {
name("parameter1")
defaultValue("")
description("")
trim(true)
}
}
def jobParametersDev = {
jobParameters
string {
name("parameter2")
defaultValue("")
description("")
trim(true)
}
string {
name("parameter3")
defaultValue("")
description("")
trim(true)
}
}
def createJob(name, env, runCommand, jobParameters) {
job("Job-${-> name}-${-> env}") {
description("My first job for ${-> name}")
parameters(jobParameters)
steps {
shell {
command(runCommand)
}
}
}
}
for (name in names) {
for (env in environments) {
if (env == 'DEV') {
def runCommand = "python35 -u ./TestSuite-${-> name}.py %parameter1% %parameter2%,%parameter3% %browser%"
createJob(name, env, runCommand, jobParametersDev)
} else {
def runCommand = "python35 -u ./TestSuite-${-> name}.py %parameter1% ${-> env} %browser%"
createJob(name, env, runCommand, jobParameters)
}
}
}
To summarise - the last thing which I tried is:
def jobParametersDev = {
jobParameters
...
}
But it doesn't work... Only values for jobParametersDev are visible.
How to add these values? If it's not necessary I don't want to double the same code.
I will be really grateful for any help.
You can not simply call one closure within another. But you can chain method calls. You just need to pass the job reference.
def generateParameters = { job ->
job.parameters {
stringParam('param3', '', '')
// more params here...
}
}
def generateDevParameters = { job ->
generateParameters(job)
job.parameters {
stringParam('param4', '', '')
// more params here...
}
}
def createJob(name, generateParameters) {
def j = job(name) {
// more config here...
}
generateParameters(j)
}
createJob('test1', generateParameters)
createJob('test2', generateDevParameters)
Related
I am building a library for myself and using ConfigSlurper to parse a config.groovy file, and other config sources into an aggregated ConfigObject that I am using Micronaught compile time IOC to inject
#Bean
#Named('config')
ConfigObject config () {
Map envMap = [:]
def sysProps = System.getenv()
if (System.getProperty("env") ) {
envMap = System.getenv().findResult { it.key?.toLowerCase().contains "env" }.collect { [(it.key?.toLowerCase().substring(0, 2)): it.value.toLowerCase()] }
} else
envMap.get ('env', "development")
def env = envMap.get('env')
def resourcePath = "src${File.separatorChar}${env =="test" ?: "main"}${File.separatorChar}resources${File.separatorChar}"
ConfigObject config = new ConfigSlurper().parse(new File("${resourcePath}ApplicationConfig.groovy").text /*.toURI().toURL()*/)
config.put('systemProperties', sysProps)
config.putAll(envMap)
config.put('projectPath', System.getProperty("user.dir"))
config.put('resourcesPath', resourcePath.toString())
File resourceDirectory = new File ("${System.getProperty("user.dir")}$File.separatorChar$resourcePath")
FilenameFilter filter = {file, name -> name.matches(~/^.*properties$/) }
List propsFiles = resourceDirectory.listFiles (filter)
filter = {file, name -> name.matches(~/^.*yaml$/) }
List yamlFiles = resourceDirectory.listFiles (filter)
propsFiles.each {file ->
Properties prop = new Properties()
prop.load(new FileInputStream (file) )
Map propsMap = [:]
for (key in prop.stringPropertyNames()) {
propsMap.put(key, prop.getProperty(key))
}
config.putAll(propsMap)
}
yamlFiles.each {file ->
def yamlConfig = new YamlSlurper().parseText(file.text)
config.putAll(yamlConfig)
}
config
}
the resources/applicationConfig.groovy file looks like this
framework {
environments {
development {
server = "local" // choice of {local|clustered}
vertxOptions {
}
}
test {
server = "local" // choice of {local|clustered}
vertxOptions {
}
}
production {
server = "clustered" // choice of {local|clustered}
vertxOptions {
}
}
}
}
The code parses the groovy file but when I look at what in the configObject, its created the tope level framework map entry - but the lower levels are not there.
calling configObject.framework in code returns the object but it is of size() == 0!
I don't understand why the rest of the structure is not getting built.
Can any one advise why the internal structure is not being parsed and built.
Can anyone suggest what I have done wrong ?
I have just done a stripped back version like this
def configText = """
framework {
environments {
development {
server = "local" // choice of {local|clustered}
vertxOptions {
}
}
test {
server = "local" // choice of {local|clustered}
vertxOptions {
}
}
production {
server = "clustered" // choice of {local|clustered}
vertxOptions {
}
}
}
}
"""
ConfigObject conf = new ConfigSlurper().parse(configText)
assert conf.framework?.environments?.size() == 3
where the error shows this
Caught: Assertion failed:
assert conf.framework?.environments?.size() == 3
| | | | |
| | [:] 0 false
| ['environments':[:]]
['framework':['environments':[:]]]
Assertion failed:
assert conf.framework?.environments?.size() == 3
| | | | |
| | [:] 0 false
| ['environments':[:]]
['framework':['environments':[:]]]
at scripts.configSlurperTest.run(configSlurperTest.groovy:33)
at
The environments section when parsed by configslurper is conditional to the environment you pass to the constructor of ConfigSlurper itself.
Lets say you want the development config to be the default, but then you want to override this when run in test or production, you would do the following:
def configText = """
framework {
// defaults
mode = 'development'
server = "local" // choice of {local|clustered}
vertxOptions {
}
environments {
// test settings
test {
mode = 'testing'
server = "local" // choice of {local|clustered}
vertxOptions {
}
}
// production settings
production {
mode = 'live!'
server = "clustered" // choice of {local|clustered}
vertxOptions {
}
}
}
}
"""
// Default values
assert new ConfigSlurper().parse(configText).framework.mode == 'development'
// Test values
assert new ConfigSlurper('test').parse(configText).framework.mode == 'testing'
// Production values
assert new ConfigSlurper('production').parse(configText).framework.mode == 'live!'
I'm developing a Jenkins shared library right now.
I wasn't able to figure how to easily "wrap" a code inside a function without copy-pasting the whole code. For example: If a developer sets a value to true, then I want to wrap the whole code inside a function. Right now I want to use this to allow e.g. the gitlabIntegration to be turned off from the Jenkinsfile.
Example:
// vars/stageWrapper.groovy
def call(Map parameters = [:], body) {
stage(stageName) {
if (pushtoGitlab) {
gitlabCommitStatus(stageName) {
if (!containerName) body()
else {
container(containerName) {
body()
}
}
}
} else {
if (!containerName) body()
else {
container(containerName) {
body()
}
}
}
}
}
let the user select if the stage should be pushed to gitlab via the gitlabCommitStatus wrapper.
switch to a specified container or use default container (if none is specified)
To realize this I currently repeat the code, which I really don't like...
Is there any way of achieving the same, but without repeating the same code over and over?
Thank You!
In Groovy you can reuse a Closure in different DSL-Builders by setting it's delegate to builder's delegate.
Something like this should work:
def containerNameBody = { body ->
if (!containerName)
body()
else
container(containerName) {
body()
}
}
def call(Map parameters = [:], body) {
stage(stageName) {
containerNameBody.delegate = delegate
if (pushtoGitlab)
gitlabCommitStatus(stageName) {
containerNameBody body
}
else
containerNameBody body
}
}
How about following approach to pass down the param into function, then decide how to do inside the function by the param value.
def gitHub(gitHubOn) {
}
def gitLab(gitLabOn) {
}
def call(Map parameters = [:], body){
//some code....
foo=bar
gitLab(parameters.gitLabOn)
gitHub(parameters.gitHubOn)
body()
}
Is there a way to define a block/environment with custom open and close methods? Currently, I have:
script {
withCredentials([usernamePassword(credentialsId: '...', usernameVariable: 'CONFIG_USER', passwordVariable: 'CONFIG_PASS')]) {
def sql = Sql.newInstance("...", CONFIG_USER, CONFIG_PASS, "com.mysql.jdbc.Driver")
sql.rows("SELECT * FROM visualization").each { row ->
println "row ${row.branch}"
}
sql.close()
}
}
I would like to be able to do:
with sqlConnection() { sql ->
sql.rows("SELECT * FROM visualization").each { row ->
println "row ${row.branch}"
}
}
Where it automatically opens/closes the connection accordingly. I am new to Groovy, so it's the syntax I'm concerned about. In Python I would do this using an object __enter__/__exit__.
If I understand you correctly you want a new method sqlConnection() that does the withCredentials part?
You can use a closure parameter to do something before or after something else.
def sqlConnection(Closure withSqlClosure) {
withCredentials([usernamePassword(credentialsId: '...', usernameVariable: 'CONFIG_USER', passwordVariable: 'CONFIG_PASS')]) {
Sql.newInstance("...", CONFIG_USER, CONFIG_PASS, "com.mysql.jdbc.Driver").withCloseable {sql ->
withSqlClosure(sql)
}
}
}
Can be used like this
sqlConnection() { sql ->
sql.rows("SELECT * FROM visualization").each { row ->
println "row ${row.branch}"
}
}
So everything before the call to the closure (withSqlClosure(sql)) corresponds to __enter__ everything after the call is your __exit__. Note that you will need to lookout for exceptions. Usually you will want to wrap the closure call in a try { ... } finally { ... } Statement. Here I used withCloseable which does that for us (assuming Sql.newInstance returns a Closeable).
To aid your IDE and enable #CompileStatic you should also add a #ClosureParams
def sqlConnection(
#ClosureParams(value = groovy.transform.stc.SimpleType,
options = ["your.sql.type"]) Closure withSqlClosure) {
withCredentials([usernamePassword(credentialsId: '...', usernameVariable: 'CONFIG_USER', passwordVariable: 'CONFIG_PASS')]) {
Sql.newInstance("...", CONFIG_USER, CONFIG_PASS, "com.mysql.jdbc.Driver").withCloseable {sql ->
withSqlClosure(sql)
}
}
}
Here your.sql.type is the return type of Sql.newInstance.
how do I print available environments from a config file? What is the form of the ojbect ConfigSlurper creates?
I tried
def config2 = new ConfigSlurper().parse(new File('obieeadmincfg.groovy').toURL())
config2.config.environments.each { println ${it} }
and
println prettyPrint(toJson(config2))
and
for ( i in 0 ..config2.config.environments.size()-1)
println config2.config.environments[i]
groovy.config
//Config3.groovy
obieadmin {
//default values
serverurl = "http://default.mycompany.com"
}
environments {
pldev01 {
obieeadmin {
serverurl = 'devgdwobi03.x.com'
}
}
plsbx02 {
obieeadmin {
serverurl = 'devgdwobi03.x.com'
}
}
}
I'm afraid you can't do this out-of box.
But using a bit of Groovy Metaprogramming it's achievable.
Groovy config slurper parses proper Groovy file, and you can do the same with GroovyShell. You can catch call to environment method providing closure in binding. In that closure you have to collect all top-level method calls(with same methodMissing).
Providing base script with property and method missing handlers, you can suppress runtime errors, and execute script without much care to other properties.
Not the nicest path, but it works.
package test
import org.codehaus.groovy.control.CompilerConfiguration
class A extends Script {
def propertyMissing(String name) { null }
def propertyMissing(String name, def arg) {}
def methodMissing(String name, def args) {}
#Override Object run() { null }
}
class NameCollector {
def envs = []
def methodMissing(String name, def args) { envs << name }
}
// configure interceptors.
def configuration = new CompilerConfiguration()
configuration.scriptBaseClass = 'test.A'
def nc = new NameCollector()
def environments = { Closure it ->
it.delegate = nc;
it.resolveStrategy = Closure.DELEGATE_ONLY
it()
}
// execute config script
new GroovyShell([environments: environments] as Binding, configuration).evaluate(new File("config.groovy"))
nc.envs // Return, print them.
Not sure if this is going to work forever but currently you can do it by overriding the setting for the 'environments' conditional block, like this:
def config = """environments {
dev {
foo = "bar"
}
prod {
foo = "baz"
}
}"""
def configSlurper = new ConfigSlurper()
configSlurper.registerConditionalBlock('environments', null)
assert configSlurper.parse(config).environments.keySet() == ['dev', 'prod'].toSet()
Let's say I have a DSL like this
setup {name = "aDSLScript"}
println "this is common groovy code"
doStuff {println "I'm doing dsl stuff"}
One would have a delegating class implementing the methods 'setup' and 'doStuff' usually. Beside, one could write common Groovy code to be executed (println...).
What I am searching for, is a way to execute this in two steps. In the first step only the setup method should be processed (neither println). The second step handles the other parts.
At the moment, I have two delegating classes. One implements 'setup' the other one implements 'doStuff'. But both execute the println statement, of course.
You can create a single class to intercept the method calls from the script and let it coordinate the following method invoke. I did it through reflection, but you can go declarative if you want. These are the model and script classes:
class FirstDelegate {
def setup(closure) { "firstDelegate.setup" }
}
class SecondDelegate {
def doStuff(closure) { "secondDelegate.doStuff" }
}
class MethodInterceptor {
def invokedMethods = []
def methodMissing(String method, args) {
invokedMethods << [method: method, args: args]
}
def delegate() {
def lookupCalls = { instance ->
def invokes = instance.metaClass.methods.findResults { method ->
invokedMethods.findResult { invocation ->
invocation.method == method.name ?
[method: method, invocation: invocation] : null
}
}
invokes.collect { invoked ->
invoked.method.invoke(instance, invoked.invocation.args)
}
}
return lookupCalls(new FirstDelegate()) + lookupCalls(new SecondDelegate())
}
}
Here be scripts and assertions:
import org.codehaus.groovy.control.CompilerConfiguration
def dsl = '''
setup {name = "aDSLScript"}
println "this is common groovy code"
doStuff {println "Ima doing dsl stuff"}
'''
def compiler = new CompilerConfiguration()
compiler.scriptBaseClass = DelegatingScript.class.name
def shell = new GroovyShell(this.class.classLoader, new Binding(), compiler)
script = shell.parse dsl
interceptor = new MethodInterceptor()
script.setDelegate interceptor
script.run()
assert interceptor.invokedMethods*.method == [ 'setup', 'doStuff' ]
assert interceptor.delegate() ==
['firstDelegate.setup', 'secondDelegate.doStuff']
Notice I didn't bothered intercepting println call, which is a DefaultGroovyMethods thus, a little more cumbersome to handle.
Also having the class MethodInterceptor implementing the method delegate() is not a good idea, since this allows the user-defined script to call it.
I found a way to split up execution of the DSL script. I used a CompilationCustomizer to remove every statement from AST except the doFirst{}. So the first run will only execute doFirst. The second run does everything else. Here's some code:
class DoFirstProcessor {
def doFirst(Closure c) {
c()
}
}
class TheRestProcessor {
def doStuff(Closure c) {
c()
}
def methodMissing(String name, args) {
//nothing to do
}
}
def dsl = "
println 'this is text that will not be printed out in first line!'
doFirst { println 'First things first: e.g. setting up environment' }
doStuff { println 'doing some stuff now' }
println 'That is it!'
"
class HighlanderCustomizer extends CompilationCustomizer {
def methodName
HighlanderCustomizer(def methodName) {
super(CompilePhase.SEMANTIC_ANALYSIS)
this.methodName = methodName
}
#Override
void call(SourceUnit sourceUnit, GeneratorContext generatorContext, ClassNode classNode) throws CompilationFailedException {
def methods = classNode.getMethods()
methods.each { MethodNode m ->
m.code.each { Statement st ->
if (!(st instanceof BlockStatement)) {
return
}
def removeStmts = []
st.statements.each { Statement bst ->
if (bst instanceof ExpressionStatement) {
def ex = bst.expression
if (ex instanceof MethodCallExpression) {
if (!ex.methodAsString.equals(methodName)) {
removeStmts << bst
}
} else {
removeStmts << bst
}
} else {
removeStmts << bst
}
}
st.statements.removeAll(removeStmts)
}
}
}
}
def cc = new CompilerConfiguration()
cc.addCompilationCustomizers new HighlanderCustomizer("doFirst")
cc.scriptBaseClass = DelegatingScript.class.name
def doFirstShell = new GroovyShell(new Binding(), cc)
def doFirstScript = doFirstShell.parse dsl
doFirstScript.setDelegate new DoFirstProcessor()
doFirstScript.run()
cc.compilationCustomizers.clear()
def shell = new GroovyShell(new Binding(), cc)
def script = shell.parse dsl
script.setDelegate new TheRestProcessor()
script.run()
I did another variation of this where I execute the DSL in one step. See my blog post about it: http://hackserei.metacode.de/?p=247