I am trying to use S3 inbound channel adapter to download files from S3. Here is my config:
s3.xml:
<int:chain input-channel="s3ReaderChannel" output-channel="uncompressPayloadChannel">
<int:service-activator ref="s3Bean" method="generateS3ObjectHash" />
<int-aws:s3-inbound-channel-adapter
bucket="${s3.bucket}"
session-factory="s3SessionFactory"
auto-create-local-directory="true"
auto-startup="false"
filename-pattern="*.gz"
local-directory="."
local-filename-generator-expression="#this"
temporary-file-suffix=".transffering"
remote-directory="/remote/mytransfers"
local-filter="acceptAllFilter"/>
</int:chain>
<bean id="s3SessionFactory"
class="org.springframework.integration.aws.support.S3SessionFactory"/>
aws-credentials.xml:
<!-- Define global credentials for all the AWS clients -->
<aws-context:context-credentials>
<aws-context:instance-profile-credentials/>
<aws-context:simple-credentials access-key="${aws.accesskey}"
secret-key="${aws.secretkey}"/>
</aws-context:context-credentials>
<!-- Define global region -->
<aws-context:context-region region="${aws.region}"/>
When I try to execute, I am getting:
Exception in thread "main"`
`org.springframework.beans.factory.BeanDefinitionStoreException: Unexpected
exception parsing XML document from file`; nested exception is
java.lang.IllegalArgumentException: 'beanName' must not be empty
at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:414)
at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:336)
...30 more
Caused by: java.lang.IllegalArgumentException: 'beanName' must not be empty
at org.springframework.util.Assert.hasText(Assert.java:181)
at org.springframework.beans.factory.config.RuntimeBeanReference.<init>(RuntimeBeanReference.java:58)
at org.springframework.beans.factory.config.RuntimeBeanReference.<init>(RuntimeBeanReference.java:46)
at org.springframework.beans.factory.support.BeanDefinitionBuilder.addPropertyReference(BeanDefinitionBuilder.java:226)
at org.springframework.integration.config.xml.AbstractPollingInboundChannelAdapterParser.doParse(AbstractPollingInboundChannelAdapterParser.java:64)
...20 more
`
From the stack trace, AbstractPollingInboundChannelAdapterParser.java:64 is about outputChannel, which I dont understand since it is in a chain.
What am I missing here?
Right, Inbound Channel Adapter is the beginning of the flow and it can't be declared in the <chain> at all. More over you have that mess like you declare it after some <int:service-activator>.
You have to move the <int-aws:s3-inbound-channel-adapter> outside of the <chain> and keep in mind that this one is going to be the start of your flow.
I'm not sure what made you think wrong way, but looks like you need more info from the Reference Manual.
Related
In my application I would like to re-use the same message transformer inside of multiple <int:chain>.
In such chains I perform http requests to different endpoints and I need to add the same basic authentication header. I would like to declare the code for adding a header only once, i.e:
<int:header-enricher id="authHeaderAdder">
<int:header expression="'Basic ' + new String(T(java.util.Base64).encoder.encode(('${http.user}' + ':' + '${http.password}').bytes))"
name="Authorization"/>
</int:header-enricher>
And then I would like to use it with ref in my chain before making http request:
<int:chain input-channel="someHttpChain">
<int:transformer ref="authHeaderAdder"/>
<http:outbound-gateway.../>
<int:transformer ref="someResponseTransformer"/>
</int:chain>
The problem is that I get an error on application startup:
Configuration problem: The 'input-channel' attribute is required for the top-level endpoint element: 'int:header-enricher' with id='authHeaderAdder'
How can I re-use authHeaderAdder without writing any java code and making a <bean/>?
You definitely need to use an input-channel on that <int:header-enricher>, e.g. input-channel="authChannel" but inside the <chain> you get a gain to use something like <int:gateway request-channel="authChannel"/>. That's all: you are reusing the same transformer, but via the Spring Integration trick with the MessageChannel.
Such an approach is cool the way that you can add more endpoint in that authChannel flow without any changes in the original flow where you use that gateway.
I need to intercept ALL the Spring Integration components at runtime and should be able to fetch the attribute values in order to log a meaningful message.
For example:
<int-http:outbound-gateway url="someURL" http-method="GET"
request-channel="channel1"
expected-response-type="com.example.Test"
message-converters="customMessageConverters">
<int-http:uri-variable name="testId" expression="headers.testId"/>
</int-http:outbound-gateway>
In the example above, I need to intercept int-http:outbound-gateway and capture the value for url, request-channel and expected-response-type. We need to do this for all http outbound gateway.
Similarly, for all other components like int-http:inbound-gateway, int-http:inbound-channel-adapter, int:transformer, int:header-enricher, int:chain, int:router, etc.
I have tried creating a custom class implementing BeanPostProcessor - postProcessAfterInitialization method. Checked for the bean name to be matching with the component, and tried to retrieve all the details but the beans are created and this method is called at the server startup itself. My requirement is to capture the flow as and when the user navigates and any particular route is being called. Also I am not able to find Java class name for all the component apart from the below. Still finding for the rest.
org.springframework.integration.http.inbound.HttpRequestHandlingMessagingGateway for int-http:inbound-gateway, org.springframework.integration.http.outbound.AbstractHttpRequestExecutingMessageHandler for int-http:outbound-gateway
Update:
I have tried the below but cant see any extra output in logs with respect to message history. Is anything missing in the above code?
<int:message-history />
<int:logging-channel-adapter id="logging"
log-full-message="true" logger-name="message.history" level="DEBUG"/>
<int:wire-tap pattern="*" order="3" channel="logging" />
or
<int:message-history />
<int:logging-channel-adapter id="logger"
log-full-message="true" logger-name="message.history" level="DEBUG"/>
<int:channel id="wiretapChannel">
<int:interceptors>
<int:wire-tap channel="logger"/>
</int:interceptors>
</int:channel>
Also, I am trying to inject LogMessage into wire-tap inorder to perform some additional tasks from MessageHistory data. But the control doesn't enter handleMessage method. Please help.
<bean id="logMessage" class="com.logging.LogMessage"/>
<int:service-activator input-channel="wiretapChannel" ref="logMessage" method="handleMessage"></int:service-activator>
public class LogMessage {
public void handleMessage(org.springframework.messaging.Message<?> message) throws MessagingException {
MessageHistory history = MessageHistory.read(message);
for (int i = 0; i < history.size(); i++) {
Properties properties = history.get(i);
getLogger().info("history: " + properties.get("name"));
}
}
}
Well, it's not so standard task, especially for the reading properties you would like to print. More over many of them are based on the SpEL expressions and the actual value depends on the request message.
There is though a component which is very close what you would like to get. It is called Message History.
All what you need is <int:message-history/> - and all the Spring Integration components will be tracked and will store they point into the MessageHistory.HEADER_NAME to show the whole path of the message over integration flow.
In addition I usually also use something like this:
<wire-tap channel="logger"/>
<logging-channel-adapter id="logger" log-full-message="true" logger-name="message.history"/>
To intercept all the channels in the application and log messages with their message history.
You may create your own POJO subscriber (<service-activator>) for the global <wire-tap> and perform some smart logic to extract a MessageHistory from the message: MessageHistory.read(Message<?>). Such a MessageHistory is a List<Properties> where, actually you can cast that Properties in the MessageHistory.Entry class and walk over its properties:
public String getName() {
return this.getProperty(NAME_PROPERTY);
}
public String getType() {
return this.getProperty(TYPE_PROPERTY);
}
public String getTimestamp() {
return this.getProperty(TIMESTAMP_PROPERTY);
}
With the name you can go to the BeanFactory to get the real component instance and already there try to extract required properties for your purpose, but again: not all of them are going to be available just because...
You also can consult with the Integration Graph for possible public properties of the IntegrationNode implementations.
i have a code
<int-jpa:updating-outbound-gateway
auto-startup="true"
native-query="update Transactions t set t.transaction_Status = :transactionStatus
where t.bank_Reference_Number = :bankReferenceNumber "
entity-manager="entityManager" persist-mode="PERSIST"
use-payload-as-parameter-source="false">
<int-jpa:transactional />
<int-jpa:parameter name="transactionStatus" expression="payload['transactionStatus']" />
<int-jpa:parameter name="bankReferenceNumber" expression="payload['bankReferenceNumber']" />
</int-jpa:updating-outbound-gateway>
i want to add error channel , so that if some exception occurs i can see that.
The error-channel is for flows which works independently of the user: Message Driven Channel Adapter, e.g. <int-jms:message-driven-channel-adapter>, where an Listener Container is started in the infinite loop and does its work actively.
Another sample is Polling Consumer, which runs the polling task periodically, e.g. <int-file:inbound-channel-adapter>, or anyone as a consumer on the QueueChannel. And again: it does that actively, independently of your existence.
The <int-jpa:updating-outbound-gateway> is passive component, therefore it can't do anything until you send a message. Therefore any error caused in this kind of components are thrown to the caller, like it is done in the raw Java method invocation.
Read more about error handling in the Reference Manual.
For your use-case you should consider to add error-channel in the upstream poller if that.
Another case is ExpressionEvaluatingRequestHandlerAdvice, which with its failureChannel may play some kind of error handling for the particular MessageHandler.
The error channel goes on some upstream component (usually whatever starts the flow).
An alternative is to add and ExpressionEvaluatingAdvice to the gateway; see the reference manual.
I have built a Spring Integration application and transferred some messages around and tried to bring them together with an Aggregator. The application reaches the Aggregator but does not deliver exactly what I want specifically I do not release the group and move onto the next step.
My problem however is my aggregator doesn't have the original message (from before the Splitter). My aggregator is defined as follows
<int:aggregator input-channel="deirBoxProcessorToAggregatorChannel"
ref="loggingAggregator" method="logAggregation"
output-channel="aggregatorToTransformer"
expire-groups-upon-completion="true"/>
And the code inside it is as follows..
public class LoggingAggregator {
private static final Logger LOGGER = Logger.getLogger(LoggingAggregator.class);
public void logAggregation(Message<File> message) {
LOGGER.info("Have aggregated messsages. Will archive");
}
My message in that method, although it enters it, is always null.
Application Context/XML Spring Integration definition
<int:splitter input-channel="transformerToSplitterChannel"
ref="fileToMessageSplitter"
output-channel="shippedSplitterToRouterChannel"
method="split" apply-sequence="true"/>
<!-- Now use a router to determine which Message builder these messages are sent onto -->
<int:router input-channel="shippedSplitterToRouterChannel"
ref="shippedToTypeRouter" />
<int:transformer input-channel="deirShippedBoxToTransformerChannel"
ref="shippedBoxTransformer" method="transform" output-
channel="deirShippedTransformerToProcessorChannel"/>
<int:service-activator id="wellFormedShippedBoxProcess"
input-channel="deirShippedTransformerToProcessorChannel"
output-channel="deirBoxProcessorToAggregatorChannel"
ref="deirShippedFileProcessor" method="processBox" />
<int:service-activator id="malformedShippedBoxProcess"
input-channel="deirMalformedShippedTransformerToProcessorChannel"
output-channel="deirBoxProcessorToAggregatorChannel"
ref="deirShippedFileProcessor"
method="processMalformedBox" />
<int:aggregator input-channel="deirBoxProcessorToAggregatorChannel"
ref="loggingAggregator" method="logAggregation"
output-channel="aggregatorToTransformer"
expire-groups-upon-completion="true"/>
<int:transformer expression="headers.file_originalFile"
input-channel="aggregatorToTransformer"
output-channel="transformerToArchiver" />
<int-file:outbound-channel-adapter id="deirArchiver"
channel="transformerToArchiver"
directory="${dataexhange.springintg.refactor.archive.dir}"
delete-source-files="true"/>
The process gets all the way to the Aggregator but does not seem to make it past to the Transformer or OutboundChannelAdapter archiver.
Thank you in advance.
Your LoggingAggregator isn't correct. I recommend you to read the Reference Manual.
Your logAggregation method should be like this:
public File logAggregation(List<String> lines) {
LOGGER.info("Have aggregated messsages. Will archive");
// Create Files from lines
return file;
}
It is a main method of Aggregator: to get a list of objects and return one object.
Artem's answer is correct. I mistakenly thought that the objects I returned to the aggregator would be of type that were sent off by the splitter. You can follow how through debugging I came to that realisation in the comments to Artem's answer.
I did see somewhere, probably in the manual you can in fact return a type that can be cast from the channel that feeds into the aggregator.
With that understanding I could in fact return Object, and cast back up to the required type for use in the logging object I would use either subsequent to or as part of the aggregator.
I'm completely at a loss with my knowledge gap.
Mule 3.3.1 CE. I must use that version for now.
I have a flow which works fine until I try to use a security filter with valid credentials.
The Mule code follows. It won't make much business sense as I've pared it down to the minimum that produces the problem. Normally the outbound call is in a separate flow, but I pulled it into the Main flow for the example.
If I have the security filter on the inbound endpoint commented out and execute this, I get the expected response of the logger message and the "foo" return payload.
<https:connector name="HTTPSConnector" validateConnections="true" sendBufferSize="0" receiveBufferSize="0" receiveBacklog="0" clientSoTimeout="10000" serverSoTimeout="10000" socketSoLinger="0" doc:name="HTTP\HTTPS">
<https:tls-key-store path="/opt/eai/common/keystore/EAIKeystore.jks" keyPassword="${key.password}" storePassword="${store.password}"/>
</https:connector>
<spring:beans>
<ss:authentication-manager alias="authManager">
<ss:authentication-provider>
<ss:user-service id="userService">
<ss:user name="PortalUser" password="password" authorities="ROLE_USER"/>
</ss:user-service>
</ss:authentication-provider>
</ss:authentication-manager>
</spring:beans>
<mule-ss:security-manager>
<mule-ss:delegate-security-provider name="memory-provider" delegate-ref="authManager"/>
</mule-ss:security-manager>
<flow name="Main" doc:name="Main">
<https:inbound-endpoint exchange-pattern="request-response" host="localhost" port="10029" path="sites/r.v1" mimeType="text/xml" encoding="UTF-8" connector-ref="HTTPSConnector">
<mule-ss:http-security-filter realm="mule-realm"/>
</https:inbound-endpoint>
<custom-transformer class="com.ca.eai.esb.transformer.site.StrategySplittingTransformer" doc:name="Split"/>
<collection-splitter/>
<https:outbound-endpoint exchange-pattern="request-response"
address="${https.outbound.account.sap-nameaddress}"
connector-ref="HTTPSConnector"
mimeType="text/xml" responseTimeout="${https.outbound.timeout}"/>
<logger level="INFO" message="GOT HERE"/>
<set-payload value="foo"/>
</flow>
The custom transformer is also pared down to the minimum:
#Override
public Object transformMessage( MuleMessage message, String outputEncoding )
{
MuleMessageCollection collection = new DefaultMessageCollection( message.getMuleContext() );
collection.addMessage( message );
return collection;
}
If I uncomment the security filter and pass in bad credentials, I get the expected security exception.
If, however, I pass in valid credentials, I get an exception that it can't serialize.
Root Exception stack trace:
java.io.NotSerializableException: org.apache.commons.httpclient.ContentLengthInputStream
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1180)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:346)
at java.util.concurrent.CopyOnWriteArrayList.writeObject(CopyOnWriteArrayList.java:857)
+ 3 more (set debug level logging or '-Dmule.verbose.exceptions=true' for everything)
Can someone tell me why adding a security filter causes this?
This is a bug that has been fixed in 3.4.x.
See this answer https://stackoverflow.com/a/17930063/387927 and the comments below it.