Hi my code works with multiple triggers and i am trying to pass specific parameters associated with
each trigger using jobDataMap.But when i am trying to assign the map in my config.groovy to
jobDataMap i get a nullpointerexception
**This is the Map in my Config.groovy-->**
Query
{
Map
{
time.'0/5 * * * * ?' = ['T1']
time.'0/10 * * * * ?' = ['T2']
templates.'T1' = ['Date','FinshDate','Location']
templates.'T2' = ['TableName']
parameterValues.'T1' = ['2014071600','2014072000','Path']
parameterValues.'T2' = ['AppleData']
}
}
**This is my Quartz Job Code for multiple triggers ->**
import org.quartz.*
import org.quartz.Trigger
import static org.quartz.JobBuilder.*;
import static org.quartz.CronScheduleBuilder.*;
import static org.quartz.TriggerBuilder.*;
import org.quartz.impl.StdSchedulerFactory;
import org.codehaus.groovy.grails.commons.GrailsApplication;
public class TrialJob
{
public static void main(String[] args)
{
String JobName
String GroupName
GrailsApplication grailsApplication;
Trigger trigger
def triggerList=[]
def jobList=[]
def cronList=["0/5 * * * * ?","0/10 * * * * ?","0/15 * * * * ?"]
// here i am creating 3 triggers which works fine
for(i in 0..2)
{
JobName="trigger"+Integer.toString(i)
GroupName = "Group"+Integer.toString(i)
println cronList[i]
JobDetail job = JobBuilder.newJob(TestJob.class).withIdentity(JobName,GroupName).build();
trigger= TriggerBuilder.newTrigger().withIdentity(JobName,GroupName).withSchedule(CronScheduleBuilder.cronSchedule(cronList[i])).build();
triggerList.add(trigger)
jobList.add(job)
}
Scheduler scheduler = new StdSchedulerFactory().getScheduler();
scheduler.start();
for(j in 0..2)
{
// here i want to put the associated parameters for each trigger in the trigger list
// For Example 1) trigger 0--> triggerList[0].jobDataMap.put(['Date','FinshDate','Location'],['2014071600','2014072000','Path'])
// 2) trigger 1--> triggerList[1].jobDataMap.put(['TableName'],['AppleData'])
scheduler.scheduleJob(jobList[j],triggerList[j]);
println "torpido"
println j
}
//while(true){};
}
public static class TestJob implements Job
{
public void execute(JobExecutionContext context) throws JobExecutionException
{
HashMap<String, String> parameter = new HashMap();
parameter=context.getMergedJobDataMap()
println "Inside Execute"
}
}
}
how do i use jobDataMap inside the above for loop (it would be more
clear by looking at the comments inside the for loop) and access them
inside the execute method ?
I'm not grails expert but it seems that grails quartz scheduler plugin should be used.
Below You can find working code:
#Grab(group='org.quartz-scheduler', module='quartz', version='2.2.1')
import org.quartz.*;
import org.quartz.impl.StdSchedulerFactory;
public class CronTriggerExample {
public static void main( String[] args ) throws Exception {
def cronExpressions = ['0/5 * * * * ?', '0/10 * * * * ?', '0/20 * * * * ?']
def triggers = cronExpressions.collect { cron ->
TriggerBuilder
.newTrigger()
.withIdentity("trigger-$cron", "trigger-$cron-group")
.withSchedule(CronScheduleBuilder.cronSchedule(cron))
.usingJobData(new JobDataMap(['cron': cron]))
.build()
}
Scheduler scheduler = new StdSchedulerFactory().getScheduler()
scheduler.start()
triggers.each { trigger ->
def job = JobBuilder.newJob(HelloJob).withIdentity("$trigger.key.name-job", "$trigger.key.name-job-group").build()
scheduler.scheduleJob(job, trigger)
}
while(true){}
}
}
public class HelloJob implements Job {
public void execute(JobExecutionContext context) throws JobExecutionException {
println "Hello Quartz with cron: ${context.mergedJobDataMap.getString('cron')}"
}
}
Job name, job group as well trigger name, trigger group must be unique. Other objects can be passed with JobDataMap. Is that clear now?
Related
I have such code:
a.b = null
a.b.c = 10 // here is NPE should be thrown
Is there way to avoid NPE by some automatic interception and initialization of b with new instance value?
In common case the object path may be long like this: a.b.c.d.e.f = 10
and any part of path may be met a null value
If you don't want to insert safe-navigation operators like a?.b?.c?.d?.e?.f = 10 you can use a macro to do this "automatically":
// I have not tried with assignment so you may need to fill a gap in the macro method
safe(a.b.c.d.e.f = 10)
// I have tested like this:
#Test
void testSafeOne() {
def objExp = null
def hashCode = safe(objExp.hashCode())
assert safe(hashCode.toString()) == null
}
#Test
void testSafeMany() {
def objExp = null
assert safe(objExp.hashCode().toString()) == null
}
#Test
void testSafeAttr() {
Integer number = null
assert safe(number.#MAX_VALUE.abs()) == null
}
#Test
void testSafeAssign() {
// TODO
}
You need to register this class through the extension mechanism:
package macros
import org.codehaus.groovy.ast.*
import org.codehaus.groovy.ast.expr.*
import org.codehaus.groovy.macro.runtime.Macro
import org.codehaus.groovy.macro.runtime.MacroContext
import static org.codehaus.groovy.ast.ClassHelper.*
import static org.codehaus.groovy.ast.tools.GeneralUtils.*
import static org.apache.groovy.ast.tools.ExpressionUtils.isThisExpression
class SafeMacroMethods {
/**
* macro version of: <code>objectExpr.?methodName()</code>
*/
#Macro
static Expression safe(MacroContext context, MethodCallExpression expression) {
if (isImplicitThis(context)) {
expression.tap { safe = true; safe(context, objectExpression) }
}
}
/**
* macro version of: <code>objectExpr.?propertyName</code>
*/
#Macro
static Expression safe(MacroContext context, PropertyExpression expression) {
if (isImplicitThis(context)) {
expression.tap {
if (getClass() == PropertyExpression) {
it.#safe = true
} else { // AttributeExpression, etc.
def field = PropertyExpression.getDeclaredField('safe')
field.accessible = true
field.set(it, true)
}
safe(context, objectExpression)
}
}
}
/**
* macro version of: <code>targetExpr ?= sourceExpr</code>
*/
#Macro
static Expression safe(MacroContext context, BinaryExpression expression) {
if (isImplicitThis(context)) {
expression.tap { safe = true; safe(context, leftExpression); safe(context, rightExpression) }
}
}
protected
static Expression safe(MacroContext context, Expression expression) {
expression
}
private static boolean isImplicitThis(MacroContext context) {
context.call.with {
isThisExpression(objectExpression) && isImplicitThis()
}
}
}
I want to use spock to find if a method in a class was called. But when I try to verify it, the when block says that the method was never called.
public class Bill {
public Bill(Integer amount) {
this.amount = amount;
}
public void pay(PaymentMethod method) {
Integer finalAmount = amount + calculateTaxes();
method.debit(finalAmount);
}
private Integer calculateTaxes() {
return 0;
}
private final Integer amount;
}
public class PaymentMethod {
public void debit(Integer amount) {
// TODO
}
public void credit(Integer amount) {
// TODO
}
}
import spock.lang.Specification
import spock.lang.Subject
class BillSpec extends Specification {
#Subject
def bill = new Bill(100)
def "Test if charge calculated"() {
given:
PaymentMethod method = Mock()
when:
bill.pay(method)
then:
1 * method.debit(100)
// 1 * bill.calculateTaxes() // Fails with (0 invocations) error
0 * _
}
}
In the example above all I want to do is verify if calculateTaxes is being called but the test fails with (0 invocations). I tried using spy but am not sure what would be the syntax since Bill takes a parameterized constructor.
You can test calculateTaxes() call when you Spy the Bill instance like this:
class SpyTestSpec extends Specification {
def "Test if charge calculated"() {
given:
def bill = Spy(new Bill(100))
PaymentMethod method = Mock()
when:
bill.pay(method)
then:
1 * method.debit(100)
1 * bill.calculateTaxes()
1 * bill.pay(method)
0 * _
}
}
Another important thing is to make calculateTaxes() method visible for the test, otherwise it will still fail:
public Integer calculateTaxes() { ... }
Note that if you want to test that nothing else was called then you should also add:
1 * bill.pay(method)
And here is the result:
I wonder if there is some integration of sleuth in hazelcast. In my application I have hazelcast queue with event listeners configured for addEntity events and problem is that span seems to be broken once this listener triggeres. I know that there is integration of sleuth for ExecutorService, but is there something similar for com.hazelcast.core.ItemListener? Thanks in advance.
UPD: Giving more details.
I have some sample service that uses both spring-cloud-sleth and hazelcast queue
package com.myapp;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;
import com.hazelcast.core.IQueue;
import com.hazelcast.core.ItemEvent;
import com.hazelcast.core.ItemListener;
import java.util.concurrent.Executors;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cloud.sleuth.DefaultSpanNamer;
import org.springframework.cloud.sleuth.TraceRunnable;
import org.springframework.cloud.sleuth.Tracer;
import org.springframework.scheduling.annotation.Async;
import org.springframework.stereotype.Service;
#Service
#Slf4j
public class SomeService {
private HazelcastInstance hazelcastInstance =
Hazelcast.newHazelcastInstance();
private IQueue<String> queue = hazelcastInstance.getQueue("someQueue");
private Tracer tracing;
#Autowired(required = false)
public void setTracer(Tracer tracer) {
this.tracing = tracer;
}
{
queue.addItemListener(new ItemListener<String>() {
#Override
public void itemAdded(ItemEvent<String> item) {
log.info("This is span");
log.info("This is item " + item);
}
#Override
public void itemRemoved(ItemEvent<String> item) {
}
}, true);
}
#Async
public void processRequestAsync() {
log.info("Processing async");
log.info("This is span");
Executors.newSingleThreadExecutor().execute(
new TraceRunnable(tracing, new DefaultSpanNamer(), () -> log.info("Some Weird stuff")));
queue.add("some stuff");
}
}
and once I call processRequestAsync I receive following output in console:
INFO [-,792a6c3ad3e91280,792a6c3ad3e91280,false] 9996 --- [nio-8080-exec-2] com.myapp.SomeController : Incoming request!
INFO [-,792a6c3ad3e91280,792a6c3ad3e91280,false] 9996 --- [nio-8080-exec-2] com.myapp.SomeController : This is current span [Trace: 792a6c3ad3e91280, Span: 792a6c3ad3e91280, Parent: null, exportable:false]
INFO [-,792a6c3ad3e91280,7d0c06d3e24a7ba1,false] 9996 --- [cTaskExecutor-1] com.myapp.SomeService : Processing async
INFO [-,792a6c3ad3e91280,7d0c06d3e24a7ba1,false] 9996 --- [cTaskExecutor-1] com.myapp.SomeService : This is span
INFO [-,792a6c3ad3e91280,8a2f0a9028f44979,false] 9996 --- [pool-1-thread-1] com.myapp.SomeService : Some Weird stuff
INFO [-,792a6c3ad3e91280,7d0c06d3e24a7ba1,false] 9996 --- [cTaskExecutor-1] c.h.i.p.impl.PartitionStateManager : [10.236.31.22]:5701 [dev] [3.8.3] Initializing cluster partition table arrangement...
INFO [-,,,] 9996 --- [e_1_dev.event-4] com.myapp.SomeService : This is span
INFO [-,,,] 9996 --- [e_1_dev.event-4] com.myapp.SomeService : This is item ItemEvent{event=ADDED, item=some stuff, member=Member [10.236.31.22]:5701 - b830dbf0-0977-42a3-a15d-800872221c84 this}
So looks like span was broked once we go to eventListener code and I wonder how can I propagate or create new span inside hazelcast queue
Sleuth (at the time of writing) does not support Hazelcast.
The solution is more general than just Hazelcast - you need to pass Zipkin's brave.Span between the client and server, but brave.Span is not serializable.
Zipkin provides a means by which to work around this.
Given a brave.Span on the client, you can convert it to a java.util.Map:
Span span = ...
Map<String, String> map = new HashMap<>();
tracing.propagation().injector(Map<String, String>::put).inject(span.context(), map);
On the server you can convert the java.util.Map back to a brave.Span:
Span span = tracer.toSpan(tracing.propagation().extractor(Map<String, String>::get).extract(map).context())
The use of java.util.Map can obviously be replaced as need be, but the principle is the same.
I can't get it to work for ItemListeners. I think we'd need to be able to wrap Hazelcast's StripedExecutor in something like a LazyTraceThreadPoolTaskExecutor (but one that accepts a plain Executor delegate instead of a ThreadPoolTaskExecutor).
For EntryProcessors, I've hacked this together. A factory to create EntryProcessors, passing in the current span from the thread that creates the processor. When the processor runs, it uses that span as the parent span in the executor thread.
#Component
public class SleuthedEntryProcessorFactory {
private final Tracer tracer;
public SleuthedEntryProcessorFactory(Tracer tracer) {
this.tracer = tracer;
}
/**
* Create an entry processor that will continue the Sleuth span of the thread
* that invokes this method.
* Mutate the given value as required. It will then be set on the entry.
*
* #param name name of the span
* #param task task to perform on the map entry
*/
public <K, V, R> SleuthedEntryProcessor<K, V, R> create(String name, Function<V, R> task) {
return new SleuthedEntryProcessor<>(name, tracer.getCurrentSpan(), task);
}
}
/**
* Copies the MDC context (which contains Sleuth's trace ID, etc.) and the current span
* from the thread that constructs this into the thread that runs this.
* #param <K> key type
* #param <V> value type
* #param <R> return type
*/
#SpringAware
public class SleuthedEntryProcessor<K, V, R> extends AbstractEntryProcessor<K, V> {
private final Map<String, String> copyOfContextMap;
private final String name;
private final Span parentSpan;
private final Function<V, R> task;
private transient Tracer tracer;
public SleuthedEntryProcessor(String name, Span parentSpan, Function<V, R> task) {
this(name, parentSpan, task, true);
}
public SleuthedEntryProcessor(
String name, Span parentSpan, Function<V, R> task, boolean applyOnBackup) {
super(applyOnBackup);
this.name = name + "Hz";
this.parentSpan = parentSpan;
this.task = task;
copyOfContextMap = MDC.getCopyOfContextMap();
}
#Override
public final R process(Map.Entry<K, V> entry) {
if (nonNull(copyOfContextMap)) {
MDC.setContextMap(copyOfContextMap);
}
Span span = tracer.createSpan(toLowerHyphen(name), parentSpan);
try {
V value = entry.getValue();
// The task mutates the value.
R result = task.apply(value);
// Set the mutated value back onto the entry.
entry.setValue(value);
return result;
} finally {
MDC.clear();
tracer.close(span);
}
}
#Autowired
public void setTracer(Tracer tracer) {
this.tracer = tracer;
}
}
Then pass the EntryProcessor to your IMap like this:
Function<V, R> process = ...;
SleuthedEntryProcessor<K, V, R> entryProcessor = sleuthedEntryProcessorFactory.create(label, process);
Map<K, R> results = iMap.executeOnEntries(entryProcessor);
I'm attempting to run this script. If I pull it all out of the class, it runs fine. When I wrap it in my "Test" class, I get this error:
Apparent variable 'json_users' was found in a static scope but doesn't
refer to a local variable, static field or class
class Test{
def testProc(JsonBuilder json_List) {
println json_List.prettyPrint
}
public static void main(String[] args){
def query = 'Select * from mytable'
def resultset = sql.rows(query)
json_users = new JsonBuilder(users:resultset)
testProc(json_users)
}
}
Any idea why I'm receiving this error?
There are multiple things wrong. One is that you cannot invoke an instance method from a static method. Another is that you can't refer to json_users without declaring it (you can in a script, but not in a class).
Instead of this:
class Test {
def testProc(JsonBuilder json_List) {
println json_List.prettyPrint
}
public static void main(String[] args){
def query = 'Select * from mytable'
def resultset = sql.rows(query)
json_users = new JsonBuilder(users:resultset)
testProc(json_users)
}
}
Try this:
class Test {
static testProc(JsonBuilder json_List) {
println json_List.prettyPrint
}
static void main(args){
def query = 'Select * from mytable'
// you will have to declare and initialize
// the sql variable.
def resultset = sql.rows(query)
def json_users = new JsonBuilder(users:resultset)
testProc(json_users)
}
}
I've written most of our project's jobs/pipelines in DSL without any previous groovy experience but now I'm stuck at more advanced problem that I can't figure out.
I'm trying to implement a method that would add 1,2,n promotions to a job.
Below you can see a fully-working method that can add one promotion, and I expected it to work in such way that I'd just call the method twice if I needed another one but then I ran into my problem - only promotion that was created the latest would be generated.
/**
* #param job DSL job object
* #param promotionName Name of the promotion
* #param nextJobs Comma seperated string of jobs to trigger when promotion is executed
* #param deployers Comma seperated string IDs that can execute promotion
* #param params Array of parameters to pass to the next job [0] = key, [1] = value.
*/
static void addPromotion(def job, String promotionName, String nextJobs, String deployers, String[][] params){
job.properties {
promotions {
promotion {
name(promotionName)
icon("star-gold")
conditions {
manual(deployers)
}
actions {
downstreamParameterized {
trigger(nextJobs) {
parameters {
for (String[] param : params){
predefinedProp(param[0]+"=",param[1])
}
}
}
}
}
}
}
}
}
The way it would work, however, if I added another 'promotion' closure like this, however, this example would generate almost identical(name and name-1) promotions:
static void addPromotion(def job, String promotionName, String nextJobs, String deployers, String[][] params){
job.properties {
promotions {
promotion {
name(promotionName)
icon("star-gold")
conditions {
manual(deployers)
}
actions {
downstreamParameterized {
trigger(nextJobs) {
parameters {
for (String[] param : params){
predefinedProp(param[0]+"=",param[1])
}
}
}
}
}
}
promotion {
name("${promotionName}-1")
icon("star-gold")
conditions {
manual(deployers)
}
actions {
downstreamParameterized {
trigger(nextJobs) {
parameters {
for (String[] param : params){
predefinedProp(param[0]+"=",param[1])
}
}
}
}
}
}
}
}
}
Is it possible to re-use closures in some way and populate the variables from a different method maybe? Or any other ideas?
This is how I solved it.
Generic promotion object part:
/**
* Adds 'promoted-builds' plugin configuration to job
**/
class Promotions {
public def job
public String promotionName
public String nextJobs
public String deployers
public String [][] params
/**
* #param job DSL job object
* #param promotionName Name of the promotion
* #param nextJobs Comma seperated string of jobs to trigger when promotion is executed
* #param deployers Comma seperated string IDs that can execute promotion
* #param params Array of parameters to pass to the next job [0] = key, [1] = value.
*/
public Promotions(Object jobName, String promotionName, String nextJobs, String deployers, String[][] params){
this.job = jobName
this.promotionName = promotionName
this.nextJobs = nextJobs
this.deployers = deployers
this.params = params
}
static void addPromotions(Promotions ... jobPromotions){
// Assuming the same job is provided as arguments
jobPromotions[0].job.properties {
promotions {
for (Promotions jobPromotion : jobPromotions){
promotion {
name(jobPromotion.promotionName)
// star-gold, star-silver
icon("star-gold")
conditions {
manual(jobPromotion.deployers)
}
actions {
downstreamParameterized {
trigger(jobPromotion.nextJobs) {
parameters {
for (String[] param : jobPromotion.params){
predefinedProp(param[0],param[1])
}
}
}
}
}
}
}
}
}
}
}
And then I prepare my params and pass them to the promotion constructor, and in the end I call addPromotions() and pass all my constructed objects to it:
def nextJobs = "${Configuration.repoName}-${branchName}-deploy-to-perf"
def deployers = "developer"
def params = [["VERSION", "\${VERSION}"],
["SOURCE_GIT_COMMIT", "\${SOURCE_GIT_COMMIT}"]] as String[][]
def promo1 = new Promotions(job, "Promote to PERF", nextJobs, deployers, params)
def nextJobs2 = "../master/${Configuration.repoName}-${branchName}-to-prod-dtr"
def deployers2 = "admin"
def params2 = [["VERSION", "\${VERSION}"],
["SOURCE_GIT_COMMIT", "\${SOURCE_GIT_COMMIT}"]] as String[][]
def promo2 = new Promotions(job, "Promote to PROD", nextJobs2, deployers2, params2)
Promotions.addPromotions(promo1, promo2)