There is an ExecutorChannel in my integration flow that will spawn one thread per message. The delegation chain looks like this:
ExecutorChannel (Spring Integration) -> BlockingExecutor (my own) -> ThreadPoolExecutor (vanilla Java)
Everything that happens in the Spring Integration part is of no interest to me. Ideally, I'd like turn off Spring Integration tracing with spring.sleuth.integration.enabled: false and simply annotate the method that will eventually called by the Spring Integration part with #NewSpan.
But when I disable spring integration, the span will appear only once: created by the main thread.
I've tried setting spring.sleuth.integration.enabled: true and excluding all but the relevant outbound channel via spring.sleuth.integration.patterns, but it's the same: only the main thread's span will appear in Zipkin.
Am I going at this from the wrong angle? What would be the best way of doing this?
The tracing propagation is done in Sleuth by the TracingChannelInterceptor. So, if you disable it, this one is not going to be applied to channels. If you see issues, then you need to apply it manually on the channel which are switching threads.
You can do that manually adding an interceptor to the channel directly or via #GlobalChannelInterceptor with respective pattern matching:
https://docs.spring.io/spring-integration/docs/current/reference/html/core.html#global-channel-configuration-interceptors
Related
Following the example found in GitHub https://github.com/spring-cloud/spring-cloud-gcp/tree/master/spring-cloud-gcp-samples/spring-cloud-gcp-pubsub-polling-binder-sample regarding polling messages from a PubSub subscription, I was wondering...
Is it possible to make a PollableMessageSource retrieve List<Message<?>> instead of a single message per poll?
I've seen the #Poller notation only being used in Source typed objects, never in Processor or Sink. Is it possible to use in such context when for example using #StreamListener or with a functional approach?
The PollableMessageSource binding and Source stream applications are fully based on the Poller and MessageSource abstraction from Spring Integration where its contract is to produce a single message to the channel configured. The point of the messaging is really to process a single message not affecting others. The failure for one message doesn't mean to fail others in the flow.
On the other hand you probably mean GCP Pub/Sub messages to be produced as a list in the Spring message payload. That is really possible, but via some custom code from Pub/Sub consumer and MessageSource impl. Although I would think twice to expect some batched from the source. Probably you may utilize an aggregator to build some small windows if your further logic is about processing as list. But again: it is going to be a single Spring message.
May be better to start thinking about a reactive function implementation where you indeed can expect a Flux<Message<?>> as an input and Spring Cloud Stream framework will take care for you how to emit the data from Pub/Sub into the reactive stream you expect.
See more info in docs: https://docs.spring.io/spring-cloud-stream/docs/3.1.0/reference/html/spring-cloud-stream.html#_reactive_functions_support
I want to use Spring Integration for HTTP inbound message processing.
I know, that it spring integration channel would run on a container thread, but if I want to use splits,
what threads would be used?
How the result of split would be returned to the initial web request thread?
(Note: I am not 100% sure if I understand you use case, but as a general remark:)
The spring integration spitter splits a message in multiple "smaller" messages. This is unrelated to multi-threading, that is, it does not per-se imply that the smaller messages are processed in parallel. It is still a sequential stream of smaller messages.
You can then process the smaller messages in parallel, by defining a handler with a given parallelism and you can define that this handler uses a dedicated thread pool.
(Sorry if this does not answer your question, please clarify).
AFAIK the Spring Cloud Stream project is based on Spring Integration. Hence I was wondering if there is a nice way to resequence a subset of inbound messages before the StreamListener handler is triggered? Or do I need to assemble the whole IntegrationFlow from scratch using XML or Java DSL config from Spring Integration?
My use case is as follows. Most of the time I process inbound messages on a Kafka topic as they come. However, a few events have to be resequenced based on CORRELATION_ID, SEQUENCE_NUMBER, and SEQUENCE_SIZE headers. In other words I'd like to keep using StreamListener as much as possible and simply plug in resequencing strategy for some events.
Yes, you would need to use Spring Integration for it. In fact Spring Cloud Stream is effectively a binding framework only. It binds message handlers to the message brokers via binders. The message handlers themselves are provided by the users.
The #StreamListener annotation is pretty much an equivalent of Spring Integration's #ServiceActivator with few extra features (e.g., conditional routing), but other then it is just a message handler.
Now, as you eluded to, you are aware that you can use Spring Integration (SI) to implement a message handler or an internal SI flow, and that is normal and recommended for complex cases.
That said, we do provide out of the box apps that implements certain EIP components and we do have, for example, and aggregator app which you can use as a starting point in implementing resequencer. Further more, given that we have an aggregator app and not resequencer, we would be glad to accept a contribution for it if you're interested.
I hope this answers you question.
i was unable to find an simple example to unit test the spring integration dsl, which involves picking up a message from queue and making a rest call.
I looked at the examples https://github.com/spring-projects/spring-integration-java-dsl but was not clear on qualifiers etc for the below code for which i want to write unit test on.
IntegrationFlows.from(Jms.inboundGateway(connectionFactory)
.id("inputChannel")
.destination(sourceQueue)
.jmsMessageConverter(new MarshallingMessageConverter(jaxbMarshaller())))
.something to validate and route
.handle(Http.outboundGateway("http://localhost:9999/create)
.httpMethod(HttpMethod.POST)
.expectedResponseType(String.class))
.get();
Something else is needed in your question to explain more the requirements.
Anyway I'll try to answer in my best feeling on the matter.
Spring Integration Java DSL is nothing more then codding tool to wire beans and build integration components into flows. In the end, at runtime, we just have a set of beans with which we can interact as with any other beans in the application context.
So, if the story is about consuming some destination from JMS and verify what we get from there, there is just enough to run ActiveMQ in the embedded mode - it is as simple as bean for:
new ActiveMQConnectionFactory("vm://localhost?broker.persistent=false")
Then you use JmsTemplate to send some test data to the desired destination (will be created on demand) and consume an Integration message from the channel defined in the mentioned in your question IntegrationFlow.
Typically for consuming test data we use a QueueChannel and its receive(long timeout). This way we block a unit test until data arrives or timeout is elapsed.
Another way to verify a flow work is with the Spring Integration Testing Framework. From there you can use a MockIntegration to replace the real MessageHandler in the application context and verify an interaction with the mock afterward.
Hope that helps a bit.
I currently am using Spring Integration to get messages off of a queue and send them to a service using a service activator. My issue is that the service I am calling requires a security context to be in place for the current thread. This can be setup by calling a no-argument method, handleAuthentication(), of another bean. I am wondering what the best way is to call this whenever a new message is received, prior to calling the service activator service? I was originally thinking I would chain together two service activators, with the first one calling handleAuthentication(), but this seems incorrect as handleAuthentication() does not require any information from the actual message.
Yes, your assumption about the security handling is correct. It is really just a side-effect aspect which should not be tied with the business logic.
Therefore we should use something which allows us to follow with the same behavior in the program. It is called as an Aspect in the programming as well.
For this purpose Spring Integration suggests a hook like MessageChannelInterceptor, where you can implement your handleAuthentication() exactly in the preReceive() callback, according to your explanation.
Another trick can be achieved with the <request-handler-advice-chain> and MethodInterceptor implementation which should populate the SecurityContext into the current thread just before target service invocation.