I am trying out to integrate Activiti with Spring Integration. The sample provided by Josh Long works but when i try to change it, specifically MessageChannels part, it starts throwing ConcurrentModificationException.
Josh Longs Activiti/Spring Integration example here.
As per Josh's code, MessageChannels is as follows:
#Configuration
class MessageChannels {
#Bean
DirectChannel requests() {
return new DirectChannel();
}
#Bean
DirectChannel replies() {
return new DirectChannel();
}
}
I changed this to spring integration (With Stream Rabbit) to
#Configuration
public interface MessageChannels {
#Input
SubscribableChannel input();
#Output("cooutput")
MessageChannel createOutput();
}
And my property file contains,
spring.cloud.stream.bindings.cooutput.destination=outputChannel
I am getting following exception, when i am calling "/start".
2016-09-10 16:14:39.087 INFO 73606 --- [ main] c.e.ActivitiIntegrationDemoApplication : Started ActivitiIntegrationDemoApplication in 8.236 seconds (JVM running for 8.759)
2016-09-10 16:15:39.524 INFO 73606 --- [nio-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring FrameworkServlet 'dispatcherServlet'
2016-09-10 16:15:39.524 INFO 73606 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : FrameworkServlet 'dispatcherServlet': initialization started
2016-09-10 16:15:39.540 INFO 73606 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : FrameworkServlet 'dispatcherServlet': initialization completed in 16 ms
2016-09-10 16:15:39.598 INFO 73606 --- [nio-8080-exec-1] o.s.i.codec.kryo.CompositeKryoRegistrar : registering [40, java.io.File] with serializer org.springframework.integration.codec.kryo.FileSerializer
2016-09-10 16:15:39.638 ERROR 73606 --- [nio-8080-exec-1] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.messaging.MessageHandlingException: error occurred in message handler [org.springframework.cloud.stream.binder.rabbit.RabbitMessageChannelBinder$SendingHandler#1b59052b]; nested exception is com.esotericsoftware.kryo.KryoException: java.util.ConcurrentModificationException
Serialization trace:
classes (sun.misc.Launcher$AppClassLoader)
beanClassLoader (org.springframework.context.event.SimpleApplicationEventMulticaster)
applicationEventMulticaster (org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext)
applicationContext (org.activiti.spring.ApplicationContextElResolver)
resolvers (org.activiti.engine.impl.javax.el.CompositeELResolver)
elResolver (org.activiti.engine.impl.el.ActivitiElContext)
cachedElContext (org.activiti.engine.impl.persistence.entity.ExecutionEntity)] with root cause
java.util.ConcurrentModificationException: null
at java.util.Vector$Itr.checkForComodification(Vector.java:1184) ~[na:1.8.0_102]
at java.util.Vector$Itr.next(Vector.java:1137) ~[na:1.8.0_102]
at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:92) ~[kryo-shaded-3.0.3.jar:na]
at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:40) ~[kryo-shaded-3.0.3.jar:na]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:552) ~[kryo-shaded-3.0.3.jar:na]
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80) ~[kryo-shaded-3.0.3.jar:na]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518) ~[kryo-shaded-3.0.3.jar:na]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:552) ~[kryo-shaded-3.0.3.jar:na]
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80) ~[kryo-shaded-3.0.3.jar:na]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518) ~[kryo-shaded-3.0.3.jar:na]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:552) ~[kryo-shaded-3.0.3.jar:na]
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80) ~[kryo-shaded-3.0.3.jar:na]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518) ~[kryo-shaded-3.0.3.jar:na]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:552) ~[kryo-shaded-3.0.3.jar:na]
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80) ~[kryo-shaded-3.0.3.jar:na]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518) ~[kryo-shaded-3.0.3.jar:na]
at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628) ~[kryo-shaded-3.0.3.jar:na]
at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:100) ~[kryo-shaded-3.0.3.jar:na]
at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:40) ~[kryo-shaded-3.0.3.jar:na]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:552) ~[kryo-shaded-3.0.3.jar:na]
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80) ~[kryo-shaded-3.0.3.jar:na]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518) ~[kryo-shaded-3.0.3.jar:na]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:552) ~[kryo-shaded-3.0.3.jar:na]
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80) ~[kryo-shaded-3.0.3.jar:na]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518) ~[kryo-shaded-3.0.3.jar:na]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:552) ~[kryo-shaded-3.0.3.jar:na]
at
Also I have added following annotations to main class.
#SpringBootApplication
#IntegrationComponentScan
#EnableBinding(MessageChannels.class)
Not sure how to help you, because your use-case is fully unclear, but you have problem with serialization for some unserializable object from the Activiti.
And looks like that one is ActivityExecution instance in the execute() method.
That Activiti-Spring Integration sample does something like this:
Start the process via /start REST endpoint.
That process make a delegation to the Spring Integration via gateway bean - ReceiveTaskActivityBehavior. It's JavaDocs says:
* A receive task is a wait state that waits for the receival of some message.
*
* Currently, the only message that is supported is the external trigger,
* given by calling the {#link RuntimeService#signal(String)} operation.
So, that execute() impl just demonstrates that we can call Spring Integration. The main part remains the same - suspend the process until signal() which is done by the MessageChannels.replies() from the /resume REST endpoint.
Now, explain, please, what you would like to achieve with the Spring Cloud Stream there. Actually both those requests()/replies() in this Activiti sample are input. You initiate something in the app via /start and /resume REST APIs.
Not sure how you have modified that sample, but that is really strange that /start initiates the message sending to the RabbitMQ Binder...
Related
I have an application that monitors FTP folder for a specific csv file foo.csv, once the file is located it pulls it to my local and generate a new output format bar.csv, the application then will send the new file finalBEY.csv back to the FTP folder and erase it from local.
Now that I have introduced a process using publishSubscribeChannel where it transforms the file to a message then uses jobLaunchingGateway that will read finalBEY.csv using batch and print it to the consol, it is not working since the finalBEY.csv is being deleted from the local forlder after sending it back to FTP, I'm using .channel("nullChannel") on the jobLaunchingGateway within the first subscribe which suppose to hold it until having a reply from the batch and then moves to the next subscribe which will send it to the ftp and remove it from local, but seems it is not the case where it is removing it from local and thus the batch is not finding finalBEY.csv and throws an error that I'm pasting below with the code.
If I remove the advice from the second subscribe it works fine as this will not delete it from local anymore.
Can you please assist on this matter?
public IntegrationFlow localToFtpFlow(Branch myBranch) {
return IntegrationFlows.from(Files.inboundAdapter(new File(myBranch.getBranchCode()))
.filter(new ChainFileListFilter<File>()
.addFilter(new RegexPatternFileListFilter("final" + myBranch.getBranchCode() + ".csv"))
.addFilter(new FileSystemPersistentAcceptOnceFileListFilter(metadataStore(dataSource), "foo"))),//FileSystemPersistentAcceptOnceFileListFilter
e -> e.poller(Pollers.fixedDelay(10_000)))
.enrichHeaders(h ->h.headerExpression("file_originalFile", "new java.io.File('"+ myBranch.getBranchCode() +"/FEFOexport" + myBranch.getBranchCode() + ".csv')",true))
.transform(p -> {
LOG.info("Sending file " + p + " to FTP branch " + myBranch.getBranchCode());
return p;
})
.log()
.transform(m -> {
this.defaultSessionFactoryLocator.addSessionFactory(myBranch.getBranchCode(),createNewFtpSessionFactory(myBranch));
LOG.info("Adding factory to delegation");
return m;
})
.publishSubscribeChannel(s ->
s.subscribe(f ->f.transform(fileMessageToJobRequest())
.handle(jobLaunchingGateway()).channel("nullChannel"))
.subscribe(h -> h.handle(Ftp.outboundAdapter(createNewFtpSessionFactory(myBranch), FileExistsMode.REPLACE)
.useTemporaryFileName(true)
.autoCreateDirectory(false)
.remoteDirectory(myBranch.getFolderPath()), e -> e.advice(expressionAdvice()))))
.get();
}
#Bean
public FileMessageToJobRequest fileMessageToJobRequest(){
FileMessageToJobRequest fileMessageToJobRequest = new FileMessageToJobRequest();
fileMessageToJobRequest.setFileParameterName("file_path");
fileMessageToJobRequest.setJob(orderJob);
return fileMessageToJobRequest;
}
#Bean
public JobLaunchingGateway jobLaunchingGateway() {
SimpleJobLauncher simpleJobLauncher = new SimpleJobLauncher();
simpleJobLauncher.setJobRepository(jobRepository);
simpleJobLauncher.setTaskExecutor(new SyncTaskExecutor());
JobLaunchingGateway jobLaunchingGateway = new JobLaunchingGateway(simpleJobLauncher);
return jobLaunchingGateway;
}
/**
* Creating the advice for routing the payload of the outbound message on different expressions (success, failure)
* #return Advice
*/
#Bean
public Advice expressionAdvice() {
ExpressionEvaluatingRequestHandlerAdvice advice = new ExpressionEvaluatingRequestHandlerAdvice();
advice.setSuccessChannelName("success.input");
advice.setOnSuccessExpressionString("payload.delete() + ' was successful'");
//advice.setOnSuccessExpressionString("inputMessage.headers['file_originalFile'].renameTo(new java.io.File(payload.absolutePath + '.success.to.send'))");
//advice.setFailureChannelName("failure.input");
advice.setOnFailureExpressionString("payload + ' was bad, with reason: ' + #exception.cause.message");
advice.setTrapException(true);
return advice;
}
Here is the error and as you can see the first lines show that it is transferred to FTP and then initiated the batch job while in the subscribe should do the batch first...
INFO 10452 --- [ask-scheduler-2] o.s.integration.ftp.session.FtpSession : File has been successfully renamed from: /ftp/erbranch/EDMS/FEFO/finalBEY.csv.writing to /ftp/erbranch/EDMS/FEFO/finalBEY.csv
Caused by: java.lang.IllegalStateException: Input resource must exist (reader is in 'strict' mode): file [C:\Java Programs\spring4ftpappftp\BEY\finalBEY.csv]
at org.springframework.batch.item.file.FlatFileItemReader.doOpen(FlatFileItemReader.java:251) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:146) ~[spring-batch-infrastructure-4.0.1.RELEASE.jar:4.0.1.RELEASE]
... 123 common frames omitted
Debugging code:
2019-07-15 10:43:02.838 INFO 4280 --- [ask-scheduler-2] o.s.integration.ftp.session.FtpSession : File has been successfully transferred to: /ftp/erbranch/EDMS/FEFO/finalBEY.csv.writing
2019-07-15 10:43:02.845 INFO 4280 --- [ask-scheduler-2] o.s.integration.ftp.session.FtpSession : File has been successfully renamed from: /ftp/erbranch/EDMS/FEFO/finalBEY.csv.writing to /ftp/erbranch/EDMS/FEFO/finalBEY.csv
2019-07-15 10:43:02.845 DEBUG 4280 --- [ask-scheduler-2] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'integrationEvaluationContext'
2019-07-15 10:43:02.848 DEBUG 4280 --- [ask-scheduler-2] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'success.input'
2019-07-15 10:43:02.849 DEBUG 4280 --- [ask-scheduler-2] o.s.integration.channel.DirectChannel : preSend on channel 'success.input', message: AdviceMessage [payload=true was successful, headers={id=eca55e1d-918e-3334-afce-66f8ab650748, timestamp=1563176582848}, inputMessage=GenericMessage [payload=BEY\finalBEY.csv, headers={file_originalFile=BEY\FEFOexportBEY.csv, id=a2f029b0-2609-1a11-67ef-4f56c7dd0752, file_name=finalBEY.csv, file_relativePath=finalBEY.csv, timestamp=1563176582787}]]
2019-07-15 10:43:02.849 DEBUG 4280 --- [ask-scheduler-2] o.s.i.t.MessageTransformingHandler : success.org.springframework.integration.transformer.MessageTransformingHandler#0 received message: AdviceMessage [payload=true was successful, headers={id=eca55e1d-918e-3334-afce-66f8ab650748, timestamp=1563176582848},
inputMessage=GenericMessage [payload=BEY\finalBEY.csv, headers={file_originalFile=BEY\FEFOexportBEY.csv, id=a2f029b0-2609-1a11-67ef-4f56c7dd0752, file_name=finalBEY.csv, file_relativePath=finalBEY.csv, timestamp=1563176582787}]]
2019-07-15 10:43:02.951 DEBUG 4280 --- [ask-scheduler-2] o.s.b.i.launch.JobLaunchingGateway : jobLaunchingGateway received message: GenericMessage [payload=JobLaunchRequest: orderJob, parameters={file_path=C:\Java Programs\spring4ftpappftp\BEY\finalBEY.csv, dummy=1563176582946}, headers={file_originalFile=BEY\FEFOexportBEY.csv, id=c98ad6cb-cced-c911-1b93-9d054baeb9d0, file_name=finalBEY.csv, file_relativePath=finalBEY.csv, timestamp=1563176582951}]
2019-07-16 08:35:29.442 INFO 10208 --- [nio-8081-exec-3] o.s.i.channel.PublishSubscribeChannel : Channel 'application.FTPOutp' has 1 subscriber(s).
2019-07-16 08:35:29.442 INFO 10208 --- [nio-8081-exec-3] o.s.i.endpoint.EventDrivenConsumer : started 1o.org.springframework.integration.config.ConsumerEndpointFactoryBean#3
2019-07-16 08:35:29.442 INFO 10208 --- [nio-8081-exec-3] o.s.i.endpoint.EventDrivenConsumer : Adding {message-handler} as a subscriber to the '1o.subFlow#1.channel#0' channel
2019-07-16 08:35:29.442 INFO 10208 --- [nio-8081-exec-3] o.s.integration.channel.DirectChannel : Channel 'application.1o.subFlow#1.channel#0' has 1 subscriber(s).
2019-07-16 08:35:29.442 INFO 10208 --- [nio-8081-exec-3] o.s.i.endpoint.EventDrivenConsumer : started 1o.subFlow#1.org.springframework.integration.config.ConsumerEndpointFactoryBean#1
2019-07-16 08:35:29.442 INFO 10208 --- [nio-8081-exec-3] o.s.i.endpoint.EventDrivenConsumer : Adding {bridge} as a subscriber to the 'FTPOutp' channel
2019-07-16 08:35:29.442 INFO 10208 --- [nio-8081-exec-3] o.s.i.channel.PublishSubscribeChannel : Channel 'application.FTPOutp' has 2 subscriber(s).
Since you are using a SyncTaskExecutor the batch job should run on the calling thread and then followed by the FTP adapter.
Use DEBUG logging and follow the message flow to see why that's not happening.
I have flow that I want to implement JdbcMetadataStore in the filers, I used SimpleMetadataStore() but this is causing issues as it an in memory store, I need to use a shared metadata store, so I have Postgres DB installed and I can see that Jdbc supports it, I declared a bean as per documents to return a JdbcMetadataStore but I'm not sure how I can use this in the filters, tried to search a lot for any example but could not find a one, note that I'm using FileSystemPersistentAcceptOnceFileListFilter as well as my datasource for Postgres is all setup in my application properties. I have pasted my code here anyone can please guide me on how to move forward?
private DataSource dataSource;
public IntegrationFlow localToFtpFlow(Branch myBranch){
return IntegrationFlows.from(Files.inboundAdapter(new File(myBranch.getBranchCode()))
.filter(new ChainFileListFilter<File>()
.addFilter(new RegexPatternFileListFilter("final" + myBranch.getBranchCode() +".csv"))
.addFilter(new FileSystemPersistentAcceptOnceFileListFilter(new SimpleMetadataStore(), "foo"))),
e -> e.poller(Pollers.fixedDelay(10_000)))
.transform( p ->{
LOG1.info("Sending file " + p + " to FTP branch " + myBranch.getBranchCode());
return p;
})
.log()
.handle(Ftp.outboundAdapter(createNewFtpSessionFactory(myBranch),FileExistsMode.REPLACE)
.useTemporaryFileName(true)
.autoCreateDirectory(false)
.remoteDirectory(myBranch.getFolderPath()))
.get();
}
public DefaultFtpSessionFactory createNewFtpSessionFactory(Branch branch){
final DefaultFtpSessionFactory factory = new DefaultFtpSessionFactory();
factory.setHost(branch.getHost());
factory.setUsername(branch.getUsern());
factory.setPort(branch.getFtpPort());
factory.setPassword(branch.getPassword());
return factory;
}
#Bean
public MetadataStore metadataStore(final DataSource dataSource) {
return new JdbcMetadataStore(dataSource);
}
error arising before I created the table manually, but shouldn't it be created automatically in Postgres as the DB is supported:
org.springframework.messaging.MessagingException: nested exception is org.springframework.jdbc.BadSqlGrammarException: PreparedStatementCallback; bad SQL grammar [INSERT INTO INT_METADATA_STORE(METADATA_KEY, METADATA_VALUE, REGION) SELECT ?, ?, ? FROM INT_METADATA_STORE WHERE METADATA_KEY=? AND REGION=? HAVING COUNT(*)=0]; nested exception is org.postgresql.util.PSQLException: ERROR: relation "int_metadata_store" does not exist
Logging info regarding the issue when adding another flow for the second server, it will trigger the first flow handling method and sending the data to ftp server for both:
Saved Branch : BEY
Hibernate: select branch0_._id as _id1_0_0_, branch0_.branch_code as branch_c2_0_0_, branch0_.folder_path as folder_p3_0_0_, branch0_.ftp_port as ftp_port4_0_0_, branch0_.host as host5_0_0_, branch0_.password as password6_0_0_, branch0_.usern as usern7_0_0_ from branch branch0_ where branch0_._id=?
BEY
2019-01-07 15:11:25.816 INFO 12940 --- [nio-8081-exec-5] o.s.integration.channel.DirectChannel : Channel 'application.intermediateChannel' has 2 subscriber(s).
2019-01-07 15:11:25.817 INFO 12940 --- [nio-8081-exec-5] o.s.i.endpoint.EventDrivenConsumer : started 1.org.springframework.integration.config.ConsumerEndpointFactoryBean#1
2019-01-07 15:11:25.817 INFO 12940 --- [nio-8081-exec-5] o.s.i.endpoint.EventDrivenConsumer : Adding {transformer} as a subscriber to the '1.channel#0' channel
2019-01-07 15:11:25.817 INFO 12940 --- [nio-8081-exec-5] o.s.integration.channel.DirectChannel : Channel 'application.1.channel#0' has 1 subscriber(s).
2019-01-07 15:11:25.817 INFO 12940 --- [nio-8081-exec-5] o.s.i.endpoint.EventDrivenConsumer : started 1.org.springframework.integration.config.ConsumerEndpointFactoryBean#0
2019-01-07 15:11:25.829 INFO 12940 --- [nio-8081-exec-5] o.s.i.e.SourcePollingChannelAdapter : started stockInboundPoller
BEY
2019-01-07 15:11:25.984 INFO 12940 --- [nio-8081-exec-5] o.s.i.endpoint.EventDrivenConsumer : Adding {message-handler} as a subscriber to the '1o.channel#2' channel
2019-01-07 15:11:25.984 INFO 12940 --- [nio-8081-exec-5] o.s.integration.channel.DirectChannel : Channel 'application.1o.channel#2' has 1 subscriber(s).
2019-01-07 15:11:25.984 INFO 12940 --- [nio-8081-exec-5] o.s.i.endpoint.EventDrivenConsumer : started 1o.org.springframework.integration.config.ConsumerEndpointFactoryBean#1
2019-01-07 15:11:25.984 INFO 12940 --- [nio-8081-exec-5] o.s.i.endpoint.EventDrivenConsumer : Adding {transformer} as a subscriber to the '1o.channel#0' channel
2019-01-07 15:11:25.984 INFO 12940 --- [nio-8081-exec-5] o.s.integration.channel.DirectChannel : Channel 'application.1o.channel#0' has 1 subscriber(s).
2019-01-07 15:11:25.984 INFO 12940 --- [nio-8081-exec-5] o.s.i.endpoint.EventDrivenConsumer : started 1o.org.springframework.integration.config.ConsumerEndpointFactoryBean#0
2019-01-07 15:11:25.984 INFO 12940 --- [nio-8081-exec-5] o.s.i.e.SourcePollingChannelAdapter : started 1o.org.springframework.integration.config.SourcePollingChannelAdapterFactoryBean#0
2019-01-07 15:11:42.655 INFO 12940 --- [ask-scheduler-4] o.s.integration.ftp.session.FtpSession : File has been successfully transferred from: /ftp/erbranch/EDMS/FEFO/FEFOexportBEY.csv
Hibernate: select branch0_._id as _id1_0_, branch0_.branch_code as branch_c2_0_, branch0_.folder_path as folder_p3_0_, branch0_.ftp_port as ftp_port4_0_, branch0_.host as host5_0_, branch0_.password as password6_0_, branch0_.usern as usern7_0_ from branch branch0_
Hibernate: insert into branch (branch_code, folder_path, ftp_port, host, password, usern) values (?, ?, ?, ?, ?, ?)
Hibernate: select currval('branch__id_seq')
Saved Branch : JNB
Hibernate: select branch0_._id as _id1_0_0_, branch0_.branch_code as branch_c2_0_0_, branch0_.folder_path as folder_p3_0_0_, branch0_.ftp_port as ftp_port4_0_0_, branch0_.host as host5_0_0_, branch0_.password as password6_0_0_, branch0_.usern as usern7_0_0_ from branch branch0_ where branch0_._id=?
JNB
2019-01-07 15:13:36.099 INFO 12940 --- [nio-8081-exec-7] o.s.integration.channel.DirectChannel : Channel 'application.intermediateChannel' has 3 subscriber(s).
2019-01-07 15:13:36.099 INFO 12940 --- [nio-8081-exec-7] o.s.i.endpoint.EventDrivenConsumer : started 2.org.springframework.integration.config.ConsumerEndpointFactoryBean#1
2019-01-07 15:13:36.099 INFO 12940 --- [nio-8081-exec-7] o.s.i.endpoint.EventDrivenConsumer : Adding {transformer} as a subscriber to the '2.channel#0' channel
2019-01-07 15:13:36.099 INFO 12940 --- [nio-8081-exec-7] o.s.integration.channel.DirectChannel : Channel 'application.2.channel#0' has 1 subscriber(s).
2019-01-07 15:13:36.099 INFO 12940 --- [nio-8081-exec-7] o.s.i.endpoint.EventDrivenConsumer : started 2.org.springframework.integration.config.ConsumerEndpointFactoryBean#0
2019-01-07 15:13:36.099 INFO 12940 --- [nio-8081-exec-7] o.s.i.e.SourcePollingChannelAdapter : started stockInboundPoller
JNB
2019-01-07 15:13:36.130 INFO 12940 --- [nio-8081-exec-7] o.s.i.endpoint.EventDrivenConsumer : Adding {message-handler} as a subscriber to the '2o.channel#2' channel
2019-01-07 15:13:36.135 INFO 12940 --- [nio-8081-exec-7] o.s.integration.channel.DirectChannel : Channel 'application.2o.channel#2' has 1 subscriber(s).
2019-01-07 15:13:36.135 INFO 12940 --- [nio-8081-exec-7] o.s.i.endpoint.EventDrivenConsumer : started 2o.org.springframework.integration.config.ConsumerEndpointFactoryBean#1
2019-01-07 15:13:36.135 INFO 12940 --- [nio-8081-exec-7] o.s.i.endpoint.EventDrivenConsumer : Adding {transformer} as a subscriber to the '2o.channel#0' channel
2019-01-07 15:13:36.135 INFO 12940 --- [nio-8081-exec-7] o.s.integration.channel.DirectChannel : Channel 'application.2o.channel#0' has 1 subscriber(s).
2019-01-07 15:13:36.135 INFO 12940 --- [nio-8081-exec-7] o.s.i.endpoint.EventDrivenConsumer : started 2o.org.springframework.integration.config.ConsumerEndpointFactoryBean#0
2019-01-07 15:13:36.135 INFO 12940 --- [nio-8081-exec-7] o.s.i.e.SourcePollingChannelAdapter : started 2o.org.springframework.integration.config.SourcePollingChannelAdapterFactoryBean#0
2019-01-07 15:13:40.981 INFO 12940 --- [ask-scheduler-1] o.s.integration.ftp.session.FtpSession : File has been successfully transferred from: /ftp/erbranch/EDMS/FEFO/FEFOexportJNB.csv
2019-01-07 15:13:46.085 INFO 12940 --- [ask-scheduler-7] o.s.i.file.FileReadingMessageSource : Created message: [GenericMessage [payload=BEY\finalBEY.csv, headers={file_originalFile=BEY\finalBEY.csv, id=42a97889-7bfb-8f77-75d8-4e7988a368f9, file_name=finalBEY.csv, file_relativePath=finalBEY.csv, timestamp=1546866826085}]]
2019-01-07 15:13:46.086 INFO 12940 --- [ask-scheduler-7] o.s.integration.handler.LoggingHandler : GenericMessage [payload=BEY\finalBEY.csv, headers={file_originalFile=BEY\finalBEY.csv, id=108a92b0-db42-620e-1c46-90652a071220, file_name=finalBEY.csv, file_relativePath=finalBEY.csv, timestamp=1546866826086}]
2019-01-07 15:13:46.160 INFO 12940 --- [ask-scheduler-8] o.s.i.file.FileReadingMessageSource : Created message: [GenericMessage [payload=JNB\finalJNB.csv, headers={file_originalFile=JNB\finalJNB.csv, id=d3b2c6a0-2e9c-42a8-c224-0ed9cbbfaabb, file_name=finalJNB.csv, file_relativePath=finalJNB.csv, timestamp=1546866826160}]]
2019-01-07 15:13:46.161 INFO 12940 --- [ask-scheduler-8] o.s.integration.handler.LoggingHandler : GenericMessage [payload=JNB\finalJNB.csv, headers={file_originalFile=JNB\finalJNB.csv, id=e34070c2-e6ff-e5e1-8c64-4af697ab1032, file_name=finalJNB.csv, file_relativePath=finalJNB.csv, timestamp=1546866826161}]
2019-01-07 15:13:47.129 INFO 12940 --- [ask-scheduler-7] o.s.integration.ftp.session.FtpSession : File has been successfully transferred to: /ftp/erbranch/EDMS/FEFO/finalBEY.csv.writing
2019-01-07 15:13:47.534 INFO 12940 --- [ask-scheduler-7] o.s.integration.ftp.session.FtpSession : File has been successfully renamed from: /ftp/erbranch/EDMS/FEFO/finalBEY.csv.writing to /ftp/erbranch/EDMS/FEFO/finalBEY.csv
2019-01-07 15:13:49.772 INFO 12940 --- [ask-scheduler-8] o.s.integration.ftp.session.FtpSession : File has been successfully transferred to: /ftp/erbranch/EDMS/FEFO/finalJNB.csv.writing
2019-01-07 15:13:50.757 INFO 12940 --- [ask-scheduler-8] o.s.integration.ftp.session.FtpSession : File has been successfully renamed from: /ftp/erbranch/EDMS/FEFO/finalJNB.csv.writing to /ftp/erbranch/EDMS/FEFO/finalJNB.csv
You can find my app here
https://github.com/EliasKhattar/Spring-Integration-Project/tree/master/spring4ftpappftp
Just use metadataSource(dataSource) instead of new SimpleMetadataStore() in the filter constructor.
EDIT
I just copied your flow into a new app (made a few changes, but not to the filter) and everything works fine for me...
#SpringBootApplication
public class So54039852Application {
public static void main(String[] args) {
SpringApplication.run(So54039852Application.class, args);
}
#Bean
public IntegrationFlow localToFtpFlow(DataSource dataSource) {
return IntegrationFlows.from(Files.inboundAdapter(new File("/tmp/foo"))
.filter(new ChainFileListFilter<File>()
.addFilter(new RegexPatternFileListFilter(".*\\.csv"))
.addFilter(new FileSystemPersistentAcceptOnceFileListFilter(metadataStore(dataSource), "foo"))),
e -> e.poller(Pollers.fixedDelay(10_000)))
.log()
.get();
}
#Bean
public ConcurrentMetadataStore metadataStore(final DataSource dataSource) {
return new JdbcMetadataStore(dataSource);
}
}
and
$ touch /tmp/foo/foo.csv
...
$ touch /tmp/foo/bar.csv
and
2019-01-09 12:46:26.332 INFO 43329 --- [ask-scheduler-2] o.s.integration.handler.LoggingHandler : GenericMessage [payload=/tmp/foo/foo.csv, headers={file_originalFile=/tmp/foo/foo.csv, id=e0613529-a657-fbd3-5e67-8bb53a58b5ca, file_name=foo.csv, file_relativePath=foo.csv, timestamp=1547055986330}]
2019-01-09 12:47:26.487 INFO 43329 --- [ask-scheduler-5] o.s.integration.handler.LoggingHandler : GenericMessage [payload=/tmp/foo/bar.csv, headers={file_originalFile=/tmp/foo/bar.csv, id=4feb74b6-d711-f028-70c7-83cdfcd0aeec, file_name=bar.csv, file_relativePath=bar.csv, timestamp=1547056046487}]
and
mysql> select * from INT_METADATA_STORE;
+---------------------+----------------+---------+
| METADATA_KEY | METADATA_VALUE | REGION |
+---------------------+----------------+---------+
| foo/tmp/foo/bar.csv | 1547056039000 | DEFAULT |
| foo/tmp/foo/foo.csv | 1547055980000 | DEFAULT |
+---------------------+----------------+---------+
2 rows in set (0.00 sec)
I don't see the file again even after a restart, but if I change the date on one of the files...
$ touch /tmp/foo/bar.csv
and
2019-01-09 12:51:58.534 INFO 44430 --- [ask-scheduler-2] o.s.integration.handler.LoggingHandler : GenericMessage [payload=/tmp/foo/bar.csv, headers={file_originalFile=/tmp/foo/bar.csv, id=f92d6b36-c948-37cc-ca56-9ef28de336f2, file_name=bar.csv, file_relativePath=bar.csv, timestamp=1547056318532}]
Using spring-integration 5.0.7 to throttle the bridging of msgs between two JMS queues.
The docs at: https://docs.spring.io/spring-integration/docs/5.0.7.RELEASE/reference/html/messaging-channels-section.html#bridge-namespace
suggest:
<int:bridge input-channel="pollable" output-channel="subscribable">
<int:poller max-messages-per-poll="10" fixed-rate="5000"/>
</int:bridge>
But schema validator complains "no nested poller allowed for subscribable input channel" on bridge elt.
But, if I put the poller on the input-channel-adapter as in:
<?xml version="1.0" encoding="UTF-8"?>
<beans:beans xmlns:int="http://www.springframework.org/schema/integration"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:beans="http://www.springframework.org/schema/beans"
xmlns:int-jms="http://www.springframework.org/schema/integration/jms"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/integration
http://www.springframework.org/schema/integration/spring-integration.xsd
http://www.springframework.org/schema/integration/jms
http://www.springframework.org/schema/integration/jms/spring-integration-jms.xsd
">
<int:channel id="inChannel" />
<int:channel id="outChannel" />
<int-jms:inbound-channel-adapter id="jmsIn" connection-factory="jmsConnectionFactory" destination-name="_dev.inQueue" channel="inChannel">
<int:poller fixed-delay="5000" max-messages-per-poll="2"/>
</int-jms:inbound-channel-adapter>
<int-jms:outbound-channel-adapter id="jmsOut" connection-factory="jmsConnectionFactory" destination-name="_dev.outQueue" channel="outChannel"/>
<int:bridge input-channel="inChannel" output-channel="outChannel">
</int:bridge>
</beans:beans>
Nothing is ever moved from input to output.
How can I bridge from one JMS queue to another with a rate-limit?
Update:
Turning on logging confirms nothing getting picked up from input channel but otherwise not helpful:
018-08-10 15:36:33.345 DEBUG 112066 --- [ask-scheduler-1] o.s.i.e.SourcePollingChannelAdapter : Received no Message during the poll, returning 'false'
2018-08-10 15:36:38.113 DEBUG 112066 --- [ask-scheduler-2] o.s.integration.jms.DynamicJmsTemplate : Executing callback on JMS Session: ActiveMQSession {id=ID:whitechapel-35247-1533940593148-3:2:1,started=true} java.lang.Object#5c278302
2018-08-10 15:36:38.116 DEBUG 112066 --- [ask-scheduler-2] o.s.i.e.SourcePollingChannelAdapter : Received no Message during the poll, returning 'false'
2018-08-10 15:36:43.115 DEBUG 112066 --- [ask-scheduler-1] o.s.integration.jms.DynamicJmsTemplate : Executing callback on JMS Session: ActiveMQSession {id=ID:whitechapel-35247-1533940593148-3:3:1,started=true} java.lang.Object#1c09a81e
2018-08-10 15:36:43.118 DEBUG 112066 --- [ask-scheduler-1] o.s.i.e.SourcePollingChannelAdapter : Received no Message during the poll, returning 'false'
Here is a Spring Boot app, using Java DSL configuration which is the exact equivalent of what you have in XML (minus the bridge); it works fine.
#SpringBootApplication
public class So51792909Application {
private static final Logger logger = LoggerFactory.getLogger(So51792909Application.class);
public static void main(String[] args) {
SpringApplication.run(So51792909Application.class, args);
}
#Bean
public ApplicationRunner runner(JmsTemplate template) {
return args -> {
for (int i = 0; i < 10; i++) {
template.convertAndSend("foo", "test");
}
};
}
#Bean
public IntegrationFlow flow(ConnectionFactory connectionFactory) {
return IntegrationFlows.from(Jms.inboundAdapter(connectionFactory)
.destination("foo"), e -> e
.poller(Pollers
.fixedDelay(5000)
.maxMessagesPerPoll(2)))
.handle(Jms.outboundAdapter(connectionFactory)
.destination("bar"))
.get();
}
#JmsListener(destination = "bar")
public void listen(String in) {
logger.info(in);
}
}
and
2018-08-10 19:38:52.534 INFO 13408 --- [enerContainer-1] com.example.So51792909Application : test
2018-08-10 19:38:52.543 INFO 13408 --- [enerContainer-1] com.example.So51792909Application : test
2018-08-10 19:38:57.566 INFO 13408 --- [enerContainer-1] com.example.So51792909Application : test
2018-08-10 19:38:57.582 INFO 13408 --- [enerContainer-1] com.example.So51792909Application : test
2018-08-10 19:39:02.608 INFO 13408 --- [enerContainer-1] com.example.So51792909Application : test
2018-08-10 19:39:02.622 INFO 13408 --- [enerContainer-1] com.example.So51792909Application : test
2018-08-10 19:39:07.640 INFO 13408 --- [enerContainer-1] com.example.So51792909Application : test
2018-08-10 19:39:07.653 INFO 13408 --- [enerContainer-1] com.example.So51792909Application : test
2018-08-10 19:39:12.672 INFO 13408 --- [enerContainer-1] com.example.So51792909Application : test
2018-08-10 19:39:12.687 INFO 13408 --- [enerContainer-1] com.example.So51792909Application : test
As you can see, the consumer gets 2 messages every 5 seconds.
Your debug log implies there are no messages in the queue.
EDIT
I figured it out; the XML parser sets the JmsTemplate receiveTimeout to nowait (-1). Since you are not using a caching connection factory, we'll never get a message because the ActiveMQ client returns immediately if there's not already a message present in the client (see this answer). Since there's no caching going on, we get a new consumer on every poll (and do a no-wait receive each time).
The DSL leaves the JmsTemplate's default (Infinite wait - which is actually wrong since it blocks the poller thread indefinitely if there are no messages).
To fix the XML version, adding receive-timeout="1000" fixes it.
However, it's better to use a CachingConnectionFactory to avoid creating a new connection/session/consumer on each poll.
Unfortunately, configurating a CachingConnectionFactory turns off Spring Boot's auto-configuration. This is fixed in Boot 2.1.
I have opened an issue to resolve the inconsistency between the DSL and XML here.
If you stick with the DSL, I would recommend setting the receive timeout to something reasonable, rather than indefinite:
#Bean
public IntegrationFlow flow(ConnectionFactory connectionFactory) {
return IntegrationFlows.from(Jms.inboundAdapter(connectionFactory)
.configureJmsTemplate(t -> t.receiveTimeout(1000))
.destination("foo"), e -> e
.poller(Pollers
.fixedDelay(5000)
.maxMessagesPerPoll(2)))
.handle(Jms.outboundAdapter(connectionFactory)
.destination("bar"))
.get();
}
But, the best solution is to use a CachingConnectionFactory.
I have a simple Gradle Spring Boot (v 1.3.3) WebMVC application I'm running from the command line via "Gradle bootrun". I am also including Spring Security and am overriding some default security configuration by including a java security config class. My build file is
buildscript {
ext {
springBootVersion = '1.3.3.RELEASE'
}
repositories {
mavenCentral()
}
dependencies {
classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
}
}
apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'spring-boot'
jar {
baseName = 'readinglist'
version = '0.0.1-SNAPSHOT'
}
sourceCompatibility = 1.8
targetCompatibility = 1.8
repositories {
mavenCentral()
}
dependencies {
compile('org.springframework.boot:spring-boot-starter-data-jpa')
compile('org.springframework.boot:spring-boot-starter-thymeleaf')
compile('org.springframework.boot:spring-boot-starter-web')
compile("org.springframework.boot:spring-boot-starter-security")
runtime('com.h2database:h2')
testCompile('org.springframework.boot:spring-boot-starter-test')
}
eclipse {
classpath {
containers.remove('org.eclipse.jdt.launching.JRE_CONTAINER')
containers 'org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/JavaSE-1.8'
}
}
task wrapper(type: Wrapper) {
gradleVersion = '2.9'
}
My security configuration class is
package readinglist;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.annotation.authentication.builders.AuthenticationManagerBuilder;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;
import org.springframework.security.core.userdetails.UserDetails;
import org.springframework.security.core.userdetails.UserDetailsService;
import org.springframework.security.core.userdetails.UsernameNotFoundException;
#Configuration
public class SecurityConfig extends WebSecurityConfigurerAdapter {
#Autowired
private ReaderRepository readerRepository;
#Override
protected void configure(HttpSecurity http) throws Exception {
http
.authorizeRequests()
.antMatchers("/readingList").access("hasRole('READER')")
.antMatchers("/**").permitAll()
.and()
.formLogin()
.loginPage("/login")
.failureUrl("/login?error=true")
.and()
.logout()
.logoutSuccessUrl("/"); // Added .and()....logoutSuccessURL()
}
#Override
protected void configure(
AuthenticationManagerBuilder auth) throws Exception {
auth
.userDetailsService(new UserDetailsService() {
#Override
public UserDetails loadUserByUsername(String username)
throws UsernameNotFoundException {
UserDetails userDetails = readerRepository.findOne(username);
if (userDetails != null) {
return userDetails;
}
throw new UsernameNotFoundException("User '" + username + "' not found.");
}
});
}
}
I also have a home controller that maps the URL "/" to the view home.html
When I run the application and go to localhost:8080/ I get the home page.
When I try to access the URL "/readingList", I get the custom login page. If I enter incorrect credentials, I return to the login page for another try. If I enter valid credentials I get the readingList page. So far so good. The problem is with the logoutSuccessURL("/"). When I go to the URL "/logout" this should log me out and take me back to "/", but instead I get the following error showing in the browser:
Whitelabel Error Page
This application has no explicit mapping for /error, so you are seeing this as a fallback.
Thu Mar 03 19:31:24 PST 2016
There was an unexpected error (type=Not Found, status=404).
No message available
I turned on debug for security and when I hit my logout link I get the following:
2016-03-03 19:48:45.033 DEBUG 22401 --- [io-8080-exec-10] o.s.web.servlet.DispatcherServlet : DispatcherServlet with name 'dispatcherServlet' processing GET request for [/error]
2016-03-03 19:48:45.033 DEBUG 22401 --- [io-8080-exec-10] s.w.s.m.m.a.RequestMappingHandlerMapping : Looking up handler method for path /error
2016-03-03 19:48:45.036 DEBUG 22401 --- [io-8080-exec-10] s.w.s.m.m.a.RequestMappingHandlerMapping : Returning handler method [public org.springframework.web.servlet.ModelAndView org.springframework.boot.autoconfigure.web.BasicErrorController.errorHtml(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse)]
2016-03-03 19:48:45.036 DEBUG 22401 --- [io-8080-exec-10] o.s.web.servlet.DispatcherServlet : Last-Modified value for [/error] is: -1
2016-03-03 19:48:45.046 DEBUG 22401 --- [io-8080-exec-10] o.s.w.s.v.ContentNegotiatingViewResolver : Requested media types are [text/html, text/html;q=0.8] based on Accept header types and producible media types [text/html])
2016-03-03 19:48:45.047 DEBUG 22401 --- [io-8080-exec-10] o.s.w.s.v.ContentNegotiatingViewResolver : Returning [org.springframework.boot.autoconfigure.web.ErrorMvcAutoConfiguration$SpelView#7ae1a0fb] based on requested media type 'text/html'
2016-03-03 19:48:45.047 DEBUG 22401 --- [io-8080-exec-10] o.s.web.servlet.DispatcherServlet : Rendering view [org.springframework.boot.autoconfigure.web.ErrorMvcAutoConfiguration$SpelView#7ae1a0fb] in DispatcherServlet with name 'dispatcherServlet'
2016-03-03 19:48:45.055 DEBUG 22401 --- [io-8080-exec-10] o.s.web.servlet.DispatcherServlet : Successfully completed request
I'm not sure why DispatcherServlet is trying to lookup "/error". I am also not being logged out since if I try to go to the URL "/readingList" again, I am not prompted for credentials.
I did a little more testing by loggin in, then going to the URL "/logout" manually. I got the following in my log:
2016-03-04 16:39:31.170 DEBUG 24395 --- [nio-8080-exec-6] o.s.security.web.FilterChainProxy : /logout reached end of additional filter chain; proceeding with original chain
2016-03-04 16:39:31.170 DEBUG 24395 --- [nio-8080-exec-6] o.s.web.servlet.DispatcherServlet : DispatcherServlet with name 'dispatcherServlet' processing GET request for [/logout]
2016-03-04 16:39:31.170 DEBUG 24395 --- [nio-8080-exec-6] s.w.s.m.m.a.RequestMappingHandlerMapping : Looking up handler method for path /logout
2016-03-04 16:39:31.171 DEBUG 24395 --- [nio-8080-exec-6] s.w.s.m.m.a.RequestMappingHandlerMapping : Did not find handler method for [/logout]
2016-03-04 16:39:31.171 DEBUG 24395 --- [nio-8080-exec-6] o.s.w.s.handler.SimpleUrlHandlerMapping : Matching patterns for request [/logout] are [/**]
2016-03-04 16:39:31.171 DEBUG 24395 --- [nio-8080-exec-6] o.s.w.s.handler.SimpleUrlHandlerMapping : URI Template variables for request [/logout] are {}
2016-03-04 16:39:31.172 DEBU G 24395 --- [nio-8080-exec-6] o.s.w.s.handler.SimpleUrlHandlerMapping : Mapping [/logout] to HandlerExecutionChain with handler [ResourceHttpRequestHandler [locations=[ServletContext resource [/], class path resource [META-INF/resources/], class path resource [resources/], class path resource [static/], class path resource [public/]], resolvers=[org.springframework.web.servlet.resource.PathResourceResolver#e146f93]]] and 1 interceptor
2016-03-04 16:39:31.172 DEBUG 24395 --- [nio-8080-exec-6] o.s.web.servlet.DispatcherServlet : Last-Modified value for [/logout] is: -1
2016-03-04 16:39:31.172 DEBUG 24395 --- [nio-8080-exec-6] o.s.web.servlet.DispatcherServlet : Null ModelAndView returned to DispatcherServlet with name 'dispatcherServlet': assuming HandlerAdapter completed request handling
2016-03-04 16:39:31.173 DEBUG 24395 --- [nio-8080-exec-6] o.s.web.servlet.DispatcherServlet : Successfully completed request
2016-03-04 16:39:31.173 DEBUG 24395 --- [nio-8080-exec-6] o.s.s.w.a.ExceptionTranslationFilter : Chain processed normally
2016-03-04 16:39:31.173 DEBUG 24395 --- [nio-8080-exec-6] s.s.w.c.SecurityContextPersistenceFilter : SecurityContextHolder now cleared, as request processing completed
2016-03-04 16:39:31.196 DEBUG 24395 --- [nio-8080-exec-6] o.s.web.servlet.DispatcherServlet : DispatcherServlet with name 'dispatcherServlet' processing GET request for [/error]
....
It seems like the "/logout" URL is not working which I don't undertand. I though the default logout URL is "/logout".
I found a solution in a similar project here, http://spr.com/part-5-integrating-spring-security-with-spring-boot-web/. In my security config I changed
.logout()
.logoutSuccessUrl("/")
.and()
...
to
.logout()
.logoutRequestMatcher(new AntPathRequestMatcher("/logout"))
.logoutSuccessUrl("/")
.and()
...
I'm not sure if this is the "preferred" solution, and I don't know why the call to logoutRequestMatcher(...) is needed. I have other (non-Spring-Boot) Spring Security projects that don't use the call to logoutRequestMatcher(...), and the .logout(...).logoutSuccessUrl(...) call works just fine.
.and()
.logout()
.invalidateHttpSession(true)
.logoutUrl("/logout")
.permitAll();
i prefer this solution because it wipe the session data so you can't access anymore try it yousrself to make sure open the developer tools in the browser and see the session !!
In the documentation, it looks like I can set up a plain text password instead of an encrypted one.
com.iplanet.am.service.secret
When using a plain text password, set this to the password for the agent profile, and leave am.encryption.pwd blank.
So, I've set the following in my OpenSSOAgentBootstrap.properties:
com.iplanet.am.service.secret = myPlainTextPassword
am.encryption.pwd =
Accessing the agent application now gives me:
java.lang.RuntimeException: Failed to load configuration: Invalid application password specified
com.sun.identity.agents.arch.AgentConfiguration.bootStrapClientConfiguration(AgentConfiguration.java:790)
com.sun.identity.agents.arch.AgentConfiguration.initializeConfiguration(AgentConfiguration.java:1140)
com.sun.identity.agents.arch.AgentConfiguration.<clinit>(AgentConfiguration.java:1579)
com.sun.identity.agents.arch.Manager.<clinit>(Manager.java:675)
com.sun.identity.agents.filter.AmAgentBaseFilter.initializeFilter(AmAgentBaseFilter.java:274)
com.sun.identity.agents.filter.AmAgentBaseFilter.getAmFilterInstance(AmAgentBaseFilter.java:364)
com.sun.identity.agents.filter.AmAgentBaseFilter.doFilter(AmAgentBaseFilter.java:73)
I also saw from here that I should be using com.iplanet.am.service.password instead. This did not work either.
Is there something that must be set on the OpenAM server? Am I missing something on the agent side of things?
Edit 1
OpenAM Server is v12.0.0, and the Tomcat agent is v3.3.0.
Log entries before the exception:
2015-04-01 12:44:09,634 [localhost-startStop-1] INFO org.springframework.web.servlet.DispatcherServlet - FrameworkServlet 'myapp': initialization started
2015-04-01 12:44:09,634 [localhost-startStop-1] INFO org.springframework.web.context.support.XmlWebApplicationContext - Refreshing WebApplicationContext for namespace 'myapp': startup date [Wed Apr 01 12:44:09 PDT 2015]; parent: Root WebApplicationContext
2015-04-01 12:44:09,634 [localhost-startStop-1] INFO org.springframework.beans.factory.xml.XmlBeanDefinitionReader - Loading XML bean definitions from ServletContext resource [/WEB-INF/servlet-context.xml]
Apr 01, 2015 12:44:09 PM org.apache.catalina.core.ApplicationContext log
INFO: Initializing Spring FrameworkServlet 'myapp'
2015-04-01 12:44:09,775 [localhost-startStop-1] INFO org.springframework.beans.factory.support.DefaultListableBeanFactory - Overriding bean definition for bean 'mvcContentNegotiationManager': replacing [Root bean: class [org.springframework.web.accept.ContentNegotiationManagerFactoryBean]; scope=; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null] with [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=org.springframework.web.servlet.config.annotation.DelegatingWebMvcConfiguration; factoryMethodName=mvcContentNegotiationManager; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [org/springframework/web/servlet/config/annotation/DelegatingWebMvcConfiguration.class]]
2015-04-01 12:44:09,775 [localhost-startStop-1] INFO org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader - Skipping bean definition for [BeanMethod:name=mvcUriComponentsContributor,declaringClass=org.springframework.web.servlet.config.annotation.WebMvcConfigurationSupport]: a definition for bean 'mvcUriComponentsContributor' already exists. This top-level bean definition is considered as an override.
2015-04-01 12:44:09,994 [localhost-startStop-1] INFO org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter - Looking for #ControllerAdvice: WebApplicationContext for namespace 'myapp': startup date [Wed Apr 01 12:44:09 PDT 2015]; parent: Root WebApplicationContext
2015-04-01 12:44:10,135 [localhost-startStop-1] INFO org.springframework.web.servlet.handler.SimpleUrlHandlerMapping - Mapped URL path [/**] onto handler of type [class org.springframework.web.servlet.resource.DefaultServletHttpRequestHandler]
2015-04-01 12:44:10,182 [localhost-startStop-1] INFO org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter - Looking for #ControllerAdvice: WebApplicationContext for namespace 'myapp': startup date [Wed Apr 01 12:44:09 PDT 2015]; parent: Root WebApplicationContext
2015-04-01 12:44:10,213 [localhost-startStop-1] INFO org.apache.tiles.access.TilesAccess - Publishing TilesContext for context: org.springframework.web.servlet.view.tiles3.SpringWildcardServletTilesApplicationContext
2015-04-01 12:44:10,244 [localhost-startStop-1] INFO org.springframework.web.servlet.DispatcherServlet - FrameworkServlet 'myapp': initialization completed in 610 ms
Apr 01, 2015 12:44:10 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-bio-8081"]
Apr 01, 2015 12:44:10 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-bio-8081"]
java.lang.RuntimeException: Invalid application password specified
at com.sun.identity.agents.arch.AgentConfiguration.setApplicationPassword(AgentConfiguration.java:1030)
at com.sun.identity.agents.arch.AgentConfiguration.bootStrapClientConfiguration(AgentConfiguration.java:720)
at com.sun.identity.agents.arch.AgentConfiguration.initializeConfiguration(AgentConfiguration.java:1140)
at com.sun.identity.agents.arch.AgentConfiguration.<clinit>(AgentConfiguration.java:1579)
at com.sun.identity.agents.arch.Manager.<clinit>(Manager.java:675)
at com.sun.identity.agents.filter.AmAgentBaseFilter.initializeFilter(AmAgentBaseFilter.java:274)
at com.sun.identity.agents.filter.AmAgentBaseFilter.getAmFilterInstance(AmAgentBaseFilter.java:364)
at com.sun.identity.agents.filter.AmAgentBaseFilter.doFilter(AmAgentBaseFilter.java:73)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
The agent's debug.out shows:
amAgentCore:04/01/2015 12:44:35:314 PM PDT: Thread[http-bio-8081-exec-1,5,main]
**********************************************
amAgentCore:04/01/2015 12:44:35:314 PM PDT: Thread[http-bio-8081-exec-1,5,main]
AgentConfiguration.setOrganizationName: organization name for realm is set to: /
amAgentCore:04/01/2015 12:44:35:314 PM PDT: Thread[http-bio-8081-exec-1,5,main]
AgentConfiguration: service resolver set to: com.sun.identity.agents.tomcat.v6.AmTomcatAgentServiceResolver
amAgentCore:04/01/2015 12:44:35:314 PM PDT: Thread[http-bio-8081-exec-1,5,main]
AgentConfiguration: service resolver reports EJBContext available: false
amAgentCore:04/01/2015 12:44:35:314 PM PDT: Thread[http-bio-8081-exec-1,5,main]
AgentConfiguration: Application User: myAgent
amSDK:04/01/2015 12:44:35:329 PM PDT: Thread[http-bio-8081-exec-1,5,main]
**********************************************
amSDK:04/01/2015 12:44:35:329 PM PDT: Thread[http-bio-8081-exec-1,5,main]
ERROR: JCEEncryption:: failed to decrypt data
java.lang.NullPointerException
at com.iplanet.services.util.JCEEncryption.pbeDecrypt(JCEEncryption.java:246)
at com.iplanet.services.util.JCEEncryption.decrypt(JCEEncryption.java:141)
at com.iplanet.services.util.Crypt.decode(Crypt.java:343)
at com.iplanet.services.util.Crypt.decryptLocal(Crypt.java:238)
at com.sun.identity.agents.arch.AM70Crypt.decrypt(AM70Crypt.java:57)
at com.sun.identity.agents.arch.AgentConfiguration.setApplicationPassword(AgentConfiguration.java:1020)
at com.sun.identity.agents.arch.AgentConfiguration.bootStrapClientConfiguration(AgentConfiguration.java:720)
at com.sun.identity.agents.arch.AgentConfiguration.initializeConfiguration(AgentConfiguration.java:1140)
at com.sun.identity.agents.arch.AgentConfiguration.<clinit>(AgentConfiguration.java:1579)
at com.sun.identity.agents.arch.Manager.<clinit>(Manager.java:675)
at com.sun.identity.agents.filter.AmAgentBaseFilter.initializeFilter(AmAgentBaseFilter.java:274)
at com.sun.identity.agents.filter.AmAgentBaseFilter.getAmFilterInstance(AmAgentBaseFilter.java:364)
at com.sun.identity.agents.filter.AmAgentBaseFilter.doFilter(AmAgentBaseFilter.java:73)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Let us look at this source code taken from GitHub: OpenRock/OpenAM
private static synchronized void setApplicationPassword() {
if (!isInitialized()) {
try {
_crypt = ServiceFactory.getCryptProvider();
if(_crypt != null) {
String encodedPass = getProperty(SDKPROP_APP_PASSWORD);
_applicationPassword = _crypt.decrypt(encodedPass);
}
} catch (Exception ex) {
logError("AgentConfiguration: Unable to create new instance of " + "Crypt class with exception ", ex);
}
if (_applicationPassword == null || _applicationPassword.trim().length() == 0) {
throw new RuntimeException("Invalid application password specified");
}
}
}
The constant SDFSFD defined as:
public static final String SDKPROP_APP_PASSWORD = "com.iplanet.am.service.secret";
As you can see AgentConfiguration reads the password and stores it into encodedPass.
So I believe the method getCryptProvider returns broken Cryptprovider (_crypt). _crypt.decrypt(encodedPass) get NullPointerException and variable _applicationPassword get never initialized then RuntimeException will thrown.
Make sure your configuration allows or defines a CryptProvider.
ok i saw your edit
NullPointerException occurs here. Try to interpret that.
final Cipher pbeCipher = cipherProvider.getCipher(); // NPE
The documentation appears to be incorrect, from what I can tell based on the source code of AgentConfiguration, the password can be only provided in an encrypted format (which also means that am.encryption.pwd should be set up properly as well.