I'm trying to write a Jenkins Job DSL script and would like to write it as declaratively / DRY-ly as possible. The Jenkins task is calling some other tasks via a MultiJob. I have Groovy that originally looks like this (everything's contained within a class because it's referenced elsewhere):
static void buildDownstream(def parentJob, String commit_a="master",
String commit_b="master") {
parentJob.with {
steps {
phase('Phase') {
job("name_1") {
prop('COMMIT_A', commit_a)
nodeLabel('NODE_LABEL', NODE_LABEL_MAP["name_1"])
killPhaseCondition('NEVER')
}
job("name_2") {
prop('COMMIT_A', commit_a)
prop('COMMIT_B', commit_b)
nodeLabel('NODE_LABEL', NODE_LABEL_MAP["name_2"])
killPhaseCondition('NEVER')
}
job("name_3") {
prop('COMMIT_A', commit_a)
prop('COMMIT_B', commit_b)
nodeLabel('NODE_LABEL', NODE_LABEL_MAP["name_3"])
killPhaseCondition('NEVER')
}
}
}
}
}
I'd like to abstract out the job creation, which contains lots of duplication. I've ended up with something strange like this:
static void buildDownstream(def parentJob, String commit_a="master",
String commit_b="master") {
parentJob.with {
steps {
phase('Phase') {
def phase = ({ owner })();
{ ->
add_node_label=true;
{ ->
commit_a = null;
def self = ({ owner })();
addJob("name_1", self).call(phase);
}
def self = ({ owner })();
addJob("name_2", self).call(phase);
addJob("name_3", self).call(phase);
}
}
}
}
}
private static Closure addJob(String job_name, Closure callingClosure) {
return { phase ->
def job_config = {
if(commit_a != null) {
prop('COMMIT_A', commit_a)
}
if(commit_b != null) {
prop('COMMIT_B', commit_b)
}
if(add_node_label == true) {
nodeLabel('NODE_LABEL', NODE_LABEL_MAP[job_name])
}
killPhaseCondition('NEVER')
}
job_config.delegate = callingClosure
job_config.resolveStrategy = Closure.DELEGATE_ONLY
phase.job(job_name, job_config)
}
}
which, probably being totally non-idiomatic Groovy (all this def self = ({ owner })() stuff doesn't sit right with me), doesn't work at all.
Basically, I want to pass all the variables in callingClosure's scope to the job_config closure without explicitly passing all of them in as arguments. (Explicitly passing a map of arguments works, but it gets unwieldy when there are lots of arguments.) How can I do this?
(P.S: Currently, Groovy is trying to resolve the commit_a variable inside job_config as coming from javaposse.jobdsl.dsl.helpers.step.PhaseContext, which I find strange; didn't I explicitly set the delegate to a closure inside that PhaseContext?)
EDIT: From another SO question, it appears that I can set phase = delegate (which defaults to owner?) instead of ({ owner })() and be fine; I don't really get this either, since job is a property of the PhaseContext, and not its parent (?)
Well, I ended up not trying to ask Groovy to implicitly resolve variables from the delegate context, and instead just passed in the parameters in a map.
static void buildDownstream(def parentJob,
String commit_a="master", String commit_b="master") {
parentJob.with {
steps {
phase('Tests') {
def params = [COMMIT_A:commit_a]
this.getTestJob(delegate, "name_1", params)
params.COMMIT_B = commit_b
this.getTestJob(delegate, "name_2", params)
this.getTestJob(delegate, "name_3", params)
continuationCondition('ALWAYS')
}
}
}
}
private static void getTestJob(def phase, String job_name,
Map properties) {
phase.job(job_name) {
properties.each { k, v -> prop(k, v) }
killPhaseCondition('NEVER')
}
}
One problem with my original method was that I was trying to access local variables in the closures, but that requires the closure to be evaluated; that turns out to be really weird, and I guess I should just not try to do that.
Related
I am trying to write my own gradle plugin and it needs to be able to configure a set of objects - how many of these objects and what they're called is up to the user.
The doco for creating custom gradle plugins with advanced customisability is quite poor. It mentions project.container() method to do this kind of thing, but I couldn't figure out how to make it work in my usecase.
This is an example of my plugin's configuration DSL as it stands:
teregrin {
terraformVersion = '0.6.6'
root("dev"){
accessKey = "flobble"
}
root("prd"){
}
}
And this is my plugin extension object that allows me to configure it:
class TeregrinPluginExtension {
boolean debug = false
boolean forceUnzip = false
String terraformVersion = null
Set<TeregrinRoot> roots = []
def root(String name, Closure c){
def newRoot = new TeregrinRoot(name)
c.setDelegate(newRoot)
c()
roots << newRoot
}
}
The extensions wired up in my plugin in the standard way:
project.extensions.create("teregrin", TeregrinPluginExtension)
This works ok, but it's a pretty ugly configuration style, not really in the style of the typical gradle DSL.
How can I change my plugin configuration DSL to be something like this:
teregrin {
terraformVersion = '0.6.6'
roots {
dev {
accessKey = "flobble"
}
prd {
}
}
}
The gradle way of implementing such DSL is by using extensions and containers:
apply plugin: SamplePlugin
whatever {
whateverVersion = '0.6.6'
conf {
dev {}
qa {}
prod {
accessKey = 'prod'
}
}
}
task printWhatever << {
println whatever.whateverVersion
whatever.conf.each { c ->
println "$c.name -> $c.accessKey"
}
}
class SamplePlugin implements Plugin<Project> {
void apply(Project project) {
project.extensions.create('whatever', SampleWhatever)
project.whatever.extensions.conf = project.container(SampleConf)
project.whatever.conf.all {
accessKey = 'dev'
}
}
}
class SampleWhatever {
String whateverVersion
}
class SampleConf {
final String name
String accessKey
SampleConf(String name) {
this.name = name
}
}
while groovy way of doing implementing such DSL is meta programming - you need to implement methodMissing in this particular case. Below is a very simple example that demonstrates how it works:
class SomeExtension {
def devConf = new SomeExtensionConf()
void methodMissing(String name, args) {
if ('dev'.equals(name)) {
def c = args[0]
c.resolveStrategy = Closure.DELEGATE_FIRST
c.delegate = devConf
c()
} else {
throw new MissingMethodException("Could not find $name method")
}
}
def getDev() {
devConf
}
}
class SomeExtensionConf {
def accessKey
}
project.extensions.create('some', SomeExtension)
some {
dev {
accessKey = 'lol'
}
}
assert 'lol'.equals(some.dev.accessKey)
Of course it has no error checking - so the args size and type of each argument need to be validated - it's omitted for the sake of brevity.
Of course there's no need to create a separate class for each configuration (I mean dev, prod, etc.). Create a single class that holds configuration and store them all in a Map where key is configuration name.
You can find a demo here.
I have the following class:
class WidgetClient {
List<Widget> getAllWidgets() {
_actuallyGetAllWidgets()
}
void saveWidget(Widget w) {
_actuallySaveWidget(w)
}
void deleteWidget(Widget w) {
_actaullyDeleteWidget(w)
}
}
This class is a client access class for a Widget Service. Unfortunately the Widget Service is not very reliable and, for reasons I can't explain, without any sort of reproducibility, is intermittently unavailable. Any time my code executes one of the WidgetClient methods (hence invoking the remote Widget Service), I would like to retry up to 5 times if the invocation produces a WidgetServiceMethodUnavailableException. Now I could do this the non-Groovy way like so:
List<Widget> getAllWidgets() {
int maxRetries = 5
int currRetries = 0
while(currRetries <= maxRetries) {
currRetries++
try {
return _actuallyGetAllWidgets()
} catch(WidgetServiceMethodUnavailableException wsmuExc) {
continue
} catch(Throwable t) {
throw t
}
}
}
But that is nasty and worse yet, I need to add that code for each method inside the WidgetClient. I'd like to see if I could define a closure where this retry logic is stored, and then somehow invoke that closure from inside each WidgetClient method. Something like:
def faultTolerant = { Closure<T> method ->
int maxRetries = 5
int currRetries = 0
while(currRetries <= maxRetries) {
currRetries++
try {
return method()
} catch(WidgetServiceMethodUnavailableException wsmuExc) {
continue
} catch(Throwable t) {
throw t
}
}
}
Now my WidgetClient can look like:
class WidgetClient {
List<Widget> getAllWidgets() {
faultTolerant(_actuallyGetAllWidgets())
}
void saveWidget(Widget w) {
faultTolerant(_actuallySaveWidget(w))
}
void deleteWidget(Widget w) {
faultTolerant(_actaullyDeleteWidget(w))
}
}
However, having never written my own Groovy closure before, I have no idea where to start. Any ideas?
Your code looks good, all you need to do is pass closures to the faultTolerant() method which call the methods you need:
class WidgetClient {
List<Widget> getAllWidgets() {
faultTolerant{_actuallyGetAllWidgets()}
}
void saveWidget(Widget w) {
faultTolerant{_actuallySaveWidget(w)}
}
void deleteWidget(Widget w) {
faultTolerant{_actaullyDeleteWidget(w)}
}
}
As your faultTolerant method takes a Closure as the final parameter, you can call it as I have shown in the code above, and this will pass the given closure (which simply calls your actually*Widget() methods) to the faultTolerantmethod.
I am relatively new to C#, maybe you could help me with this.
I got a couple of methods callServiceXY(param1, param2, ...) that call a certain service. For many reasons these service calls can go wrong (and I don't really care for the reason in the end). So basically I need to always wrap them with something like this - to have them execute again if something goes wrong:
var i = 3;
while(i>0)
try{
call...()
} catch{
i--;
}
i=0;
}
I'd rather write this code only once. Could I somehow have a method like tryXtimes(int x, callService()) that allows me to execute an undefined or anonymous method? (I have Javascript in mind where this is possible...)?
Yes this is possible. C# 3.5 added support for Action and Func<T> types. An Action won't return any value, a Func will always return a value.
You have several different versions that also accept a number of parameters. The following Console Applications describes how you could do this:
using System;
namespace Stackoverflow
{
class Service
{
public int MyMethod() { return 42; }
public void MyMethod(string param1, bool param2) { }
public int MyMethod(object paramY) { return 42; }
}
class Program
{
static void ExecuteWithRetry(Action action)
{
try
{
action();
}
catch
{
action();
}
}
static T ExecuteWithRetry<T>(Func<T> function)
{
try
{
return function();
}
catch
{
return function();
}
}
static void Main(string[] args)
{
Service s = new Service();
ExecuteWithRetry(() => s.MyMethod("a", true));
int a = ExecuteWithRetry(() => s.MyMethod(1));
int b = ExecuteWithRetry(() => s.MyMethod(true));
}
}
}
As you can see, there are two overloads for ExecuteWithRetry. One returning void, one returning a type. You can call ExecuteWithRetry by passing an Action or a Func.
--> Edit: Awesome! Just a little extra code to complete the example:
With anonymous function/method:
ExecuteWithRetry(() =>
{
logger.Debug("test");
});
And with more parameters (action, int)
Method header:
public static void ExecuteWithRetryX(Action a, int x)
Method call:
ExecuteWithRetryX(() => { logger.Debug("test"); }, 2);
I would use the strategy/factory pattern(s) for this. This answer https://stackoverflow.com/a/13641801/626442 gives and example of the use of the strategy/factory pattern with links. The question at the above link will give you another type of example where this pattern can be adopted.
There are great examples of these design patterns here and the following are detailed intros to the Strategy pattern and the Factory pattern. The former of the last two links also shows you how to combine the two to do something like what you require.
I hope this helps.
Try following
void CallServiceXY(params object []objects)
{
Console.WriteLine("a");
throw new Exception("");
}
void Retry(int maxRetryCount, Action<object[]> action, params object[] obj)
{
int retryCount = 1;
while ( retryCount <= maxRetryCount)
{
try
{
action(obj);
return;
}
catch
{
retryCount++;
}
}
}
void Main()
{
Retry(2,CallServiceXY);
Retry(2,CallServiceXY,"");
Retry(2,CallServiceXY,"","");
}
Demo here
Trick is Action<object[]> that accepts object array and return void and params keyword in Retry method.
To return non void value, Change Action<object[]> to Func<T, object[]>.
I'd like to create a DSL with syntax like:
Graph.make {
foo {
bar()
definedMethod1() // isn't missing!
}
baz()
}
Where when the handler for this tree encounters the outermost closure, it creates an instance of some class, which has some defined methods and also its own handler for missing methods.
I figured this would be easy enough with some structure like:
public class Graph {
def static make(Closure c){
Graph g = new Graph()
c.delegate = g
c()
}
def methodMissing(String name, args){
println "outer " + name
ObjImpl obj = new ObjImpl(type: name)
if(args.length > 0 && args[0] instanceof Closure){
Closure closure = args[0]
closure.delegate = obj
closure()
}
}
class ObjImpl {
String type
def methodMissing(String name, args){
println "inner " + name
}
def definedMethod1(){
println "exec'd known method"
}
}
}
But the methodMissing handler interprets the entire closure inside Graph rather than delegating the inner closure to ObjImpl, yielding output:
outer foo
outer bar
exec'd known method
outer baz
How do I scope the missing method call for the inner closure to the inner object that I create?
The easy answer is to set the inner closure's resolveStrategy to "delegate first", but doing that when the delegate defines a methodMissing to intercept all method calls has the effect of making it impossible to define a method outside the closure and call it from inside, e.g.
def calculateSomething() {
return "something I calculated"
}
Graph.make {
foo {
bar(calculateSomething())
definedMethod1()
}
}
To allow for this sort of pattern it's better to leave all the closures as the default "owner first" resolve strategy, but have the outer methodMissing be aware of when there is an inner closure in progress and hand back down to that:
public class Graph {
def static make(Closure c){
Graph g = new Graph()
c.delegate = g
c()
}
private ObjImpl currentObj = null
def methodMissing(String name, args){
if(currentObj) {
// if we are currently processing an inner ObjImpl closure,
// hand off to that
return currentObj.invokeMethod(name, args)
}
println "outer " + name
if(args.length > 0 && args[0] instanceof Closure){
currentObj = new ObjImpl(type: name)
try {
Closure closure = args[0]
closure()
} finally {
currentObj = null
}
}
}
class ObjImpl {
String type
def methodMissing(String name, args){
println "inner " + name
}
def definedMethod1(){
println "exec'd known method"
}
}
}
With this approach, given the above DSL example, the calculateSomething() call will pass up the chain of owners and reach the method defined in the calling script. The bar(...) and definedMethod1() calls will go up the chain of owners and get a MissingMethodException from the outermost scope, then try the delegate of the outermost closure, ending up in Graph.methodMissing. That will then see that there is a currentObj and pass the method call back down to that, which in turn will end up in ObjImpl.definedMethod1 or ObjImpl.methodMissing as appropriate.
If your DSL can be nested more than two levels deep then you'll need to keep a stack of "current objects" rather than a single reference, but the principle is exactly the same.
An alternative approach might be to make use of groovy.util.BuilderSupport, which is designed for tree building DSLs like yours:
class Graph {
List children
void addChild(ObjImpl child) { ... }
static Graph make(Closure c) {
return new GraphBuilder().build(c)
}
}
class ObjImpl {
List children
void addChild(ObjImpl child) { ... }
String name
void definedMethod1() { ... }
}
class GraphBuilder extends BuilderSupport {
// the various forms of node builder expression, all of which
// can optionally take a closure (which BuilderSupport handles
// for us).
// foo()
public createNode(name) { doCreate(name, [:], null) }
// foo("someValue")
public createNode(name, value) { doCreate(name, [:], value) }
// foo(colour:'red', shape:'circle' [, "someValue"])
public createNode(name, Map attrs, value = null) {
doCreate(name, attrs, value)
}
private doCreate(name, attrs, value) {
if(!current) {
// root is a Graph
return new Graph()
} else {
// all other levels are ObjImpl, but you could change this
// if you need to, conditioning on current.getClass()
def = new ObjImpl(type:name)
current.addChild(newObj)
// possibly do something with attrs ...
return newObj
}
}
/**
* By default BuilderSupport treats all method calls as node
* builder calls. Here we change this so that if the current node
* has a "real" (i.e. not methodMissing) method that matches
* then we call that instead of building a node.
*/
public Object invokeMethod(String name, Object args) {
if(current?.respondsTo(name, args)) {
return current.invokeMethod(name, args)
} else {
return super.invokeMethod(name, args)
}
}
}
The way BuilderSupport works, the builder itself is the closure delegate at all levels of the DSL tree. It calls all its closures with the default "owner first" resolve strategy, which means that you can define a method outside the DSL and call it from inside, e.g.
def calculateSomething() {
return "something I calculated"
}
Graph.make {
foo {
bar(calculateSomething())
definedMethod1()
}
}
but at the same time any calls to methods defined by ObjImpl will be routed to the current object (the foo node in this example).
There are at least two problems with this approach:
Defining ObjImpl within the same context as Graph means that any missingMethod call will hit Graph first
Delegation appears to happen locally unless a resolveStrategy is set, e.g.:
closure.resolveStrategy = Closure.DELEGATE_FIRST
I'm implementing in Groovy a DSL for some existing file format.
In this format we have a construct like
group basic_test {
test vplan_testing {
dir: global_storage;
};
};
And here I have problem with this dir: global_storage - groovy considers "dir:" as a label, so I can't handle it.
Do you have an idea how I can receive some callback (getProperty, invokeMissingMethod) for this construct?
Thank you!
I don't believe you can achieve that this way, you need to change your dsl a bit to be able to capture that information. Here's how you could achieve that:
class Foo {
static plan = {
vplan_testing {
dir 'global_storage'
}
}
}
def closure = Foo.plan
closure.delegate = this
closure()
def methodMissing(String name, Object args) {
println "$name $args"
if(args[0] instanceof Closure)
args[0].call()
}
The output will be
dir [global_storage]
or you could defined you dsl this way:
class Foo {
static plan = {
vplan_testing {
test dir:'global_storage'
}
}
}
replace "test" by something meaningful to you domain. In this case the output would be
test [[dir:global_storage]]
Hope this helps
-ken