Helo Everyone,
i am working on spring cloud data flow
i created the application and it is working fine but we have one requirement like it will be able to start/stop whenever we need..
if i keep autoStartup="false" it is not starting at first,but i dont know how to start or stop after that.
Most of the places having only xml config.
Tried some code with some online articles but it didn't work.
can anyone know how to resolve this and if is there any example means it will be really helpful.
Actually if autoStartup is false and using CommandLineRunner i can able to start the service.but the same thing if i tried with rest endpointit is throwing error.
Below is the code snippet.
below snippet is from my code.
package com.javatechie.service;
import com.javatechie.impl.ProductBuilder;
import com.javatechie.TbeSource;
import com.javatechie.model.Product;
import com.javatechie.stopping.StopPollingAdvice;
import lombok.SneakyThrows;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.CommandLineRunner;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Scope;
import org.springframework.integration.annotation.EndpointId;
import org.springframework.integration.annotation.InboundChannelAdapter;
import org.springframework.integration.annotation.Poller;
import org.springframework.integration.annotation.ServiceActivator;
import org.springframework.integration.channel.DirectChannel;
import org.springframework.integration.config.EnableIntegration;
import org.springframework.integration.config.ExpressionControlBusFactoryBean;
import org.springframework.integration.core.MessageSource;
import org.springframework.integration.dsl.IntegrationFlow;
import org.springframework.integration.dsl.IntegrationFlows;
import org.springframework.integration.handler.LoggingHandler;
import org.springframework.integration.scheduling.PollerMetadata;
import org.springframework.messaging.Message;
import org.springframework.messaging.MessageChannel;
import org.springframework.messaging.support.GenericMessage;
import org.springframework.messaging.support.MessageBuilder;
import org.springframework.scheduling.support.PeriodicTrigger;
import org.springframework.stereotype.Component;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
import java.util.Arrays;
import java.util.List;
import java.util.function.Supplier;
#Component
#EnableIntegration
#RestController
public class ProcessingCl {
#Autowired
ApplicationContext ac;
#Bean
#Scope("singleton")
private ProductBuilder dataAccess() {
return new ProductBuilder();
}
#Bean
#EndpointId("inboundtest")
#InboundChannelAdapter(channel = TbeSource.PR1, poller = #Poller(fixedDelay = "100", errorChannel = "errorchannel"),autoStartup = "false")
public Supplier<Product> getProductSource(ProductBuilder dataAccess)
{
return ()->dataAccesss.getNext();
}
#Bean
MessageChannel controlChannel() {
return new DirectChannel();
}
#Bean
#ServiceActivator(inputChannel = "controlChannel")
ExpressionControlBusFactoryBean controlBus() {
ExpressionControlBusFactoryBean expressionControlBusFactoryBean = new ExpressionControlBusFactoryBean();
return expressionControlBusFactoryBean;
}
#Bean
CommandLineRunner commandLineRunner(#Qualifier("controlChannel") MessageChannel controlChannel) {
return (String[] args) -> {
System.out.println("Starting incoming file adapter: ");
boolean sent = controlChannel.send(new GenericMessage<>("#inboundtest.start()"));
System.out.println("Sent control message successfully? " + sent);
while (System.in.available() == 0) {
Thread.sleep(50);
}
};
}
#GetMapping("/stop11")
void test() {
controlChannel().send(new GenericMessage<>("#inboundtest.stop()"));
System.out.println("it is in call method to stop");
}
#GetMapping("/start11")
void test1() {
controlChannel().send(new GenericMessage<>("#inboundtest.start()"));
System.out.println("it is in call method to start");
}
}
Following is the error message.
Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.messaging.MessageDeliveryException: Dispatcher has no subscribers for channel 'unknown.channel.name'.; nested exception is org.springframework.integration.MessageDispatchingException: Dispatcher has no subscribers, failedMessage=GenericMessage [payload=#inboundtest.start(), headers={id=82c74865-2f68-c192-7d48-501bf3b28e02, timestamp=1608643036650}], failedMessage=GenericMessage [payload=#inboundtest.start(), headers={id=82c74865-2f68-c192-7d48-501bf3b28e02, timestamp=1608643036650}]] with root cause
org.springframework.integration.MessageDispatchingException: Dispatcher has no subscribers
The control bus, #EndpointId and command as #inboundtest.start() is the way to go.
Only the problem that you don't show what is a subscriber for that TbeSource.PR1 channel.
That's probably where you are missing the point of Spring Integration: you start producing messages from that polling channel adapter, but there is no something like service activator having that TbeSource.PR1 channel as input.
I see you say it is Data Flow, but it is not clear from your description and configuration where is the binding for a TbeSource.PR1 destination. Do you have something like #EnableBinding and respective Source definition?
Related
I am working on the integration of a Vendor software that uses for interface a DB table mimicking a queue.
Here is the JPA entity representation of such table:
#Data
#Entity
#Table(name = "TASK")
public static class Task implements Serializable {
#Id
#GeneratedValue(generator = "TaskId")
#SequenceGenerator(name = "TaskId", sequenceName = "TASK_SEQ", allocationSize = 50)
#Column(name = "ID")
private BigInteger id;
private Status status;
private LocalDate processedDate;
public enum Status {
NEW, PROCESSED, ERROR
}
}
I would like to use Spring integration to poll NEW records from this table, handle them (typical use case is to transfom and post on a JMS queue), and then udpate the record with either:
status PROCESSED if everything went fine
status ERROR if an exception occured
I tried to do so with JpaExecutor + JpaPollingChannelAdapter without much success. How would you recommend to tackle this case ?
Here is how I started:
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.integration.config.EnableIntegration;
import org.springframework.integration.dsl.IntegrationFlow;
import org.springframework.integration.dsl.IntegrationFlows;
import org.springframework.integration.dsl.Pollers;
import org.springframework.integration.handler.GenericHandler;
import org.springframework.integration.jpa.core.JpaExecutor;
import org.springframework.integration.jpa.inbound.JpaPollingChannelAdapter;
import org.springframework.stereotype.Component;
#Component
#EnableIntegration
public class ExampleJob {
#Autowired
EntityManager entityManager;
#Bean
IntegrationFlow taskExecutorFlow() {
JpaExecutor selectExecutor = new JpaExecutor(entityManager);
selectExecutor.setJpaQuery("from Task where status = 'NEW'");
JpaPollingChannelAdapter adapter = new JpaPollingChannelAdapter(selectExecutor);
return IntegrationFlows
.from(adapter, c -> c.poller(Pollers.fixedDelay(Duration.ofMinutes(5))).autoStartup(true))
.handle((task, headers) -> task)
.get();
}
See a Jpa.outboundAdapter() as the next .handle() in your flow. This one is going to perform a MERGE on the entity in the message payload.
See more in docs: https://docs.spring.io/spring-integration/docs/current/reference/html/jpa.html#jpa-outbound-channel-adapter
The poller() could be configured with a transaction to have the whole flow against retrieved entity transactional.
i'm trying to make Unit Test testing a simple GET controller method apply MockMvc.perform method but when the controller receive a request the method Mockito.when seems to doesn't manage the method call of MenuService and the test throw an exception. The Exception says menuServiceMock is null
I'm working with Mockito MockMvc JUnit
import org.junit.runner.RunWith;
import org.junit.jupiter.api.Test;
import org.mockito.Mockito;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
import org.springframework.test.context.web.WebAppConfiguration;
import org.springframework.test.web.servlet.MockMvc;
import edu.AG.LandingPageSanpietro.domain.Menu;
import edu.AG.LandingPageSanpietro.service.MenuService;
import java.util.Arrays;
import static org.hamcrest.Matchers.*;
import static org.hamcrest.Matchers.is;
import static org.mockito.Mockito.*;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.*;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.model;
#RunWith(SpringJUnit4ClassRunner.class)
#WebAppConfiguration
class MenuControllerTest {
private MockMvc mockMvc;
#Autowired
private MenuService menuServiceMock;
#Test
public void testHomeController1() throws Exception {
Menu first=new Menu("titolo1","descrizione1","filename1");
Menu second=new Menu("titolo2","descrizione2","filename2");
Mockito.when(menuServiceMock.getMenus()).thenReturn(Arrays.asList(first, second));
mockMvc.perform(get("/manageMenu"))
.andExpect(status().isOk())
.andExpect(view().name("manageMenu"))
.andExpect(forwardedUrl("/src/main/resources/tamplates/manageMenu.html"))
.andExpect(model().attribute("menus", hasSize(2)));
}
My Controller
#GetMapping("/manageMenu")
public String chiamataGestisciMenu(Model model) {
model.addAttribute("menus", menuService.getMenus());
return "manageMenu";
}
The error
java.lang.NullPointerException: Cannot invoke "edu.AG.LandingPageSanpietro.service.MenuService.getMenus()" because "this.menuServiceMock" is null
at edu.AG.LandingPageSanpietro.controller.MenuControllerTest.testHomeController1(MenuControllerTest.java:44)
I can't understand why when() method doesn't manage my menuServiceMock.getMenus() request for returning the specified list.
Use the annotation #Mock from mockito instead of #Autowired.
Seems like you have not initialized MockMvc. Try Autowiring it or initialize it in #Before method:
#RunWith(SpringJUnit4ClassRunner.class)
#WebMvcTest(controllers = ControllerToBeTested.class)
class MenuControllerTest {
#Autowired
private MockMvc mockMvc;
...
or you can even initialize it in #Before lik this:
#Before
public void setup() {
this.mockMvc = MockMvcBuilders.standaloneSetup(new ControllerToBeTested()).build();
}
I am trying to use spring-integration-file to poll a directory and create a reactive stream from files placed in this directory. This is working for the most part, but when I place a file but have no subscriber in place I get an exception. To demonstrate the problem I have written a small demo application:
import org.reactivestreams.Publisher;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.http.MediaType;
import org.springframework.integration.dsl.IntegrationFlows;
import org.springframework.integration.dsl.Pollers;
import org.springframework.integration.file.dsl.Files;
import org.springframework.messaging.Message;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
import reactor.core.publisher.Flux;
import java.io.File;
#SpringBootApplication
#RestController
public class DemoApplication {
public static void main(String[] args) {
SpringApplication.run(DemoApplication.class, args);
}
#Bean
public Publisher<Message<File>> reactiveSource() {
return IntegrationFlows
.from(Files.inboundAdapter(new File("."))
.patternFilter("*.csv"),
e -> e.poller(Pollers.fixedDelay(1000)))
.channel("processFileChannel")
.toReactivePublisher();
}
#GetMapping(value = "/files", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<String> files() {
return Flux.from(reactiveSource())
.map(message -> message.getPayload().getAbsolutePath());
}
}
So if I now do a curl to localhost:8080/files and then place a csv file in the root directory of the project everything is fine, I see the path of the file as response to my curl. But when I don't do a curl and then place a file in the root directory I get the following exception:
java.lang.IllegalStateException: The [bean 'reactiveSource.channel#0'; defined in: 'com.example.demo.DemoApplication';
from source: 'bean method reactiveSource'] doesn't have subscribers to accept messages
at org.springframework.util.Assert.state(Assert.java:97)
at org.springframework.integration.channel.FluxMessageChannel.doSend(FluxMessageChannel.java:61)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:570)
... 38 more
I thought one of the attributes of reactive streams was that when there was no subscriber the stream would not start due to the stream being lazy. But apparently this is not the case. Could someone explain what I would need to do to have the stream not start if there is no subscriber?
If you use one of the latest version, then you can use a FluxMessageChannel channel instead of that DirectChannel for the "processFileChannel". This way a SourcePollingChannel adapter will becomes reactive and indeed the source is not going to be polled until a subscription happens to that FluxMessageChannel.
You then create a Flux in your files() API from this FluxMessageChannel - no need in the .toReactivePublisher().
See more in docs: https://docs.spring.io/spring-integration/docs/current/reference/html/reactive-streams.html#source-polling-channel-adapter
The point is that .toReactivePublisher() just makes an integration flow as a Publisher exactly at this point. Everything before this point is in regular, imperative way and works independently from the downstream logic.
UPDATE
Something like this:
#Bean
FluxMessageChannel filesChannel() {
return new FluxMessageChannel();
}
#Bean
public IntegrationFlow reactiveSource() {
return IntegrationFlows
.from(Files.inboundAdapter(new File("."))
.patternFilter("*.csv"),
e -> e.poller(Pollers.fixedDelay(1000)))
.channel(filesChannel())
.get();
}
#GetMapping(value = "/files", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<String> files() {
return Flux.from(filesChannel())
.map(message -> ((File) message.getPayload()).getAbsolutePath());
}
mockito-1.10.19
powermock-mockito-1.7.1
powermock-1.7.4
junit 4.12
I have a class that has multiple constructors (java). Once constructor calls the other. I want to mock only 1 of the constructors (the one that is called from the other). I cannot change the code unfortunately - I am just testing it. Here is the class to be tested:
import java.io.File;
import java.sql.connection;
public class Foo {
public Foo (Connection connection){
this(connection, new File ());
}
public Foo (Connection connection, File file){
// do stuff
}
// other methods
}
Here is the test class I have written:
import java.io.File;
import java.sql.connection;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.Mock;
import org.mockito.Matchers;
import org.mockito.Mockito;
import org.powermock.api.mockito.PowerMockito;
import org.powermock.core.classloader.annotations.PowerMockIgnore;
import org.powermock.core.classloader.annotations.PrepareForTest;
import org.powermock.modules.junit4.legacy.PowerMockRunner;
#RunWith(PowerMockRunner.class)
#PrepareForTest(Foo.class)
#PowerMockIgnore("javax.management.*")
public class FooTest {
#Test
public void testFoo() throws Exception {
Connection mockConnection = Mockito.mock(Connection.class);
Foo fooObj = Mockito.mock(Foo.class);
PowerMockito.whenNew(Foo.class).withArguments(Matchers.notNull(), Matchers.notNull()).thenReturn(fooObj);
Foo newFooObj = new Foo (mockConnection);
assertNotNull ("newFooObj should not be null", newFooObj);
}
}
The problem is that Foo(Connection) is not being entered. Is there something I am missing?
I tried your code with the latest 1.7.x version of Powermock (1.7.4) and it works as you wanted it to. So you might just need to upgrade a few minor versions.
We've got a rather complicated spring-integration-amqp use case in one of our production applications and we've been seeing some "org.springframework.integration.MessageDispatchingException: Dispatcher has no subscribers" exceptions on startup. After the initial errors on startup, we don't see those exceptions anymore from the same components. This is seeming like some kind of startup race condition on components that depend on AMQP outbound adapters and that end up using them early in the lifecycle.
I can reproduce this by calling a gateway that sends to a channel wired to an outbound adapter in a PostConstruct method.
config:
package gadams;
import org.springframework.amqp.core.Queue;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.integration.annotation.IntegrationComponentScan;
import org.springframework.integration.dsl.IntegrationFlow;
import org.springframework.integration.dsl.IntegrationFlows;
import org.springframework.integration.dsl.amqp.Amqp;
import org.springframework.integration.dsl.channel.MessageChannels;
import org.springframework.messaging.MessageChannel;
#SpringBootApplication
#IntegrationComponentScan
public class RabbitRace {
public static void main(String[] args) {
SpringApplication.run(RabbitRace.class, args);
}
#Bean(name = "HelloOut")
public MessageChannel channelHelloOut() {
return MessageChannels.direct().get();
}
#Bean
public Queue queueHello() {
return new Queue("hello.q");
}
#Bean(name = "helloOutFlow")
public IntegrationFlow flowHelloOutToRabbit(RabbitTemplate rabbitTemplate) {
return IntegrationFlows.from("HelloOut").handle(Amqp.outboundAdapter(rabbitTemplate).routingKey("hello.q"))
.get();
}
}
gateway:
package gadams;
import org.springframework.integration.annotation.Gateway;
import org.springframework.integration.annotation.MessagingGateway;
#MessagingGateway
public interface HelloGateway {
#Gateway(requestChannel = "HelloOut")
void sendMessage(String message);
}
component:
package gadams;
import javax.annotation.PostConstruct;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.DependsOn;
import org.springframework.stereotype.Component;
#Component
#DependsOn("helloOutFlow")
public class HelloPublisher {
#Autowired
private HelloGateway helloGateway;
#PostConstruct
public void postConstruct() {
helloGateway.sendMessage("hello");
}
}
In my production use case, we have a component with a PostConstruct method where we're using a TaskScheduler to schedule a bunch of components with some that depend on AMQP outbound adapters, and some of those end up executing immediately. I've tried putting bean names on the IntegrationFlows that involve an outbound adapter and using #DependsOn on the beans that use the gateways and/or the gateway itself, but that doesn't get rid of the errors on startup.
That everything called Lifecycle. Any Spring Integration endpoints start listen for or produce messages only when their start() is performed.
Typically for standard default autoStartup = true it is done in the ApplicationContext.finishRefresh(); as a
// Propagate refresh to lifecycle processor first.
getLifecycleProcessor().onRefresh();
To start producing messages to the channel from the #PostConstruct (afterPropertiesSet()) is really very early, because it is does far away from the finishRefresh().
You really should reconsider your producing logic and that implementation into SmartLifecycle.start() phase.
See more info in the Reference Manual.