I have followed some links to create multi-tenancy in my JHipster application,but its not working.
https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#multitenacy
https://www.baeldung.com/hibernate-5-multitenancy
https://github.com/jgasmi/jhipster-mt/
I did following:
Added following in my application-dev.yml:
datasource:
type: com.zaxxer.hikari.HikariDataSource
url: jdbc:mysql://localhost:3306/myApp?useUnicode=true&characterEncoding=utf8&useSSL=false&useLegacyDatetimeCode=false&serverTimezone=UTC&createDatabaseIfNotExist=true
username: root
password: root
serverName: localhost
hikari:
poolName: Hikari
auto-commit: false
data-source-properties:
cachePrepStmts: true
prepStmtCacheSize: 250
prepStmtCacheSqlLimit: 2048
useServerPrepStmts: true
jpa:
database-platform: org.hibernate.dialect.MySQL5InnoDBDialect
database: MYSQL
show-sql: true
properties:
hibernate.id.new_generator_mappings: true
hibernate.cache.use_query_cache: false
hibernate.generate_statistics: true
hibernate.cache.use_minimal_puts: true
hibernate.cache.hazelcast.use_lite_member: true
hibernate.tenant_identifier_resolver : com.mycompany.myapp.multitenancy.TenantIdentifierResolver
hibernate.multi_tenant_connection_provider : com.mycompany.myapp.multitenancy.SchemaMultiTenantConnectionProviderImpl
hibernate.multiTenancy : SCHEMA
My TenantIdentifierResolver class:
public class TenantIdentifierResolver implements CurrentTenantIdentifierResolver {
#Override
public String resolveCurrentTenantIdentifier() {
return "myapp2";
}
#Override
public boolean validateExistingCurrentSessions() {
return true;
}
}
And my class SchemaMultiTenantConnectionProviderImpl looks as follows:
public class SchemaMultiTenantConnectionProviderImpl implements MultiTenantConnectionProvider, ServiceRegistryAwareService
{
private final Logger log = LoggerFactory.getLogger(SchemaMultiTenantConnectionProviderImpl.class);
DataSource dataSource;
#Override
public Connection getAnyConnection() throws SQLException {
return this.dataSource.getConnection();
}
#Override
public void releaseAnyConnection(Connection connection) throws SQLException {
try {
connection.createStatement().execute("USE myapp2;");
}
catch (SQLException e) {
throw new HibernateException("Could not alter JDBC connection to specified schema [public]", e);
}
connection.close();
}
#Override
public Connection getConnection(String tenantIdentifier) throws SQLException {
log.warn("Get Connection for Tenatd is:"+tenantIdentifier);
final Connection connection = getAnyConnection();
/*
* try { connection.setCatalog(tenantIdentifier); //
* connection.createStatement().execute("USE " + tenantIdentifier + ";"); //
* connection.setCatalog(tenantIdentifier); //
* connection.setSchema(tenantIdentifier); } catch (SQLException e) { throw new
* HibernateException("Could not alter JDBC connection to specified schema [" +
* tenantIdentifier + "]", e); }
*/
return connection;
}
#Override
public void releaseConnection(String tenantIdentifier, Connection connection) throws SQLException {
// try {
// connection.createStatement().execute("USE "+tenantIdentifier+";");
// }
// catch (SQLException e) {
// throw new HibernateException("Could not alter JDBC connection to specified schema [public]", e);
// }
connection.close();
}
#Override
public boolean supportsAggressiveRelease() {
return false;
}
#Override
public boolean isUnwrappableAs(Class unwrapType) {
return false;
}
#Override
public <T> T unwrap(Class<T> unwrapType) {
return null;
}
#Override
public void injectServices(ServiceRegistryImplementor serviceRegistry) {
Map lSettings = serviceRegistry.getService(ConfigurationService.class).getSettings();
DataSource localDs = (DataSource) lSettings.get("hibernate.connection.datasource");
dataSource = localDs;
}
}
I am just trying to see first that it should work, and then I can change it to be based on tenant logic, schema tenancy logic or database tenancy logic.
When I run the application following is the logs:
D:\java-project\jhipsterptest-eclipse-workspace\sample\multitennnacy>mvnw -e
[INFO] Error stacktraces are turned on.
[INFO] Scanning for projects...
[INFO]
[INFO] --------------------------< com.myapp:my-app >--------------------------
[INFO] Building My App 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] >>> spring-boot-maven-plugin:2.4.7:run (default-cli) > test-compile # my-app >>>
[INFO]
[INFO] --- maven-resources-plugin:3.2.0:copy-resources (default-resources) # my-app ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Using 'UTF-8' encoding to copy filtered properties files.
[INFO] Copying 4 resources
[INFO] Copying 19 resources
[INFO]
[INFO] --- maven-resources-plugin:3.2.0:resources (default-resources) # my-app ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Using 'UTF-8' encoding to copy filtered properties files.
[INFO] Copying 4 resources
[INFO] Copying 19 resources
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M3:enforce (enforce-versions) # my-app ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M3:enforce (enforce-dependencyConvergence) # my-app ---
[WARNING]
Dependency convergence error for net.java.dev.jna:jna:5.2.0 paths to dependency are:
+-com.myapp:my-app:0.0.1-SNAPSHOT
+-org.testcontainers:mysql:1.15.3
+-org.testcontainers:jdbc:1.15.3
+-org.testcontainers:database-commons:1.15.3
+-org.testcontainers:testcontainers:1.15.3
+-org.rnorth.visible-assertions:visible-assertions:2.1.2
+-net.java.dev.jna:jna:5.2.0
and
+-com.myapp:my-app:0.0.1-SNAPSHOT
+-org.testcontainers:mysql:1.15.3
+-org.testcontainers:jdbc:1.15.3
+-org.testcontainers:database-commons:1.15.3
+-org.testcontainers:testcontainers:1.15.3
+-com.github.docker-java:docker-java-transport-zerodep:3.2.8
+-net.java.dev.jna:jna:5.8.0
[WARNING]
Dependency convergence error for org.wildfly.common:wildfly-common:1.5.2.Final paths to dependency are:
+-com.myapp:my-app:0.0.1-SNAPSHOT
+-org.springframework.boot:spring-boot-starter-undertow:2.4.7
+-io.undertow:undertow-core:2.2.8.Final
+-org.jboss.xnio:xnio-api:3.8.0.Final
+-org.wildfly.common:wildfly-common:1.5.2.Final
and
+-com.myapp:my-app:0.0.1-SNAPSHOT
+-org.springframework.boot:spring-boot-starter-undertow:2.4.7
+-io.undertow:undertow-core:2.2.8.Final
+-org.jboss.xnio:xnio-api:3.8.0.Final
+-org.wildfly.client:wildfly-client-config:1.0.1.Final
+-org.wildfly.common:wildfly-common:1.2.0.Final
[WARNING]
Dependency convergence error for org.apiguardian:apiguardian-api:1.1.0 paths to dependency are:
+-com.myapp:my-app:0.0.1-SNAPSHOT
+-org.springframework.boot:spring-boot-starter-test:2.4.7
+-org.junit.jupiter:junit-jupiter:5.7.2
+-org.junit.jupiter:junit-jupiter-api:5.7.2
+-org.apiguardian:apiguardian-api:1.1.0
and
+-com.myapp:my-app:0.0.1-SNAPSHOT
+-org.springframework.boot:spring-boot-starter-test:2.4.7
+-org.junit.jupiter:junit-jupiter:5.7.2
+-org.junit.jupiter:junit-jupiter-api:5.7.2
+-org.junit.platform:junit-platform-commons:1.7.2
+-org.apiguardian:apiguardian-api:1.1.0
and
+-com.myapp:my-app:0.0.1-SNAPSHOT
+-org.springframework.boot:spring-boot-starter-test:2.4.7
+-org.junit.jupiter:junit-jupiter:5.7.2
+-org.junit.jupiter:junit-jupiter-params:5.7.2
+-org.apiguardian:apiguardian-api:1.1.0
and
+-com.myapp:my-app:0.0.1-SNAPSHOT
+-org.springframework.boot:spring-boot-starter-test:2.4.7
+-org.junit.jupiter:junit-jupiter:5.7.2
+-org.junit.jupiter:junit-jupiter-engine:5.7.2
+-org.apiguardian:apiguardian-api:1.1.0
and
+-com.myapp:my-app:0.0.1-SNAPSHOT
+-com.tngtech.archunit:archunit-junit5-engine:0.19.0
+-com.tngtech.archunit:archunit-junit5-engine-api:0.19.0
+-org.junit.platform:junit-platform-engine:1.7.2
+-org.apiguardian:apiguardian-api:1.1.0
and
+-com.myapp:my-app:0.0.1-SNAPSHOT
+-org.zalando:problem-spring-web:0.26.2
+-org.zalando:problem-violations:0.26.2
+-org.apiguardian:apiguardian-api:1.1.0
and
+-com.myapp:my-app:0.0.1-SNAPSHOT
+-org.zalando:problem-spring-web:0.26.2
+-org.zalando:problem-spring-common:0.26.2
+-org.apiguardian:apiguardian-api:1.1.0
and
+-com.myapp:my-app:0.0.1-SNAPSHOT
+-org.zalando:problem-spring-web:0.26.2
+-org.apiguardian:apiguardian-api:1.1.0
and
+-com.myapp:my-app:0.0.1-SNAPSHOT
+-org.zalando:problem-spring-web:0.26.2
+-org.zalando:faux-pas:0.8.0
+-org.apiguardian:apiguardian-api:1.0.0
[WARNING] Rule 0: org.apache.maven.plugins.enforcer.DependencyConvergence failed with message:
Failed while enforcing releasability. See above detailed error message.
[INFO]
[INFO] --- jacoco-maven-plugin:0.8.7:prepare-agent (pre-unit-tests) # my-app ---
[INFO] argLine set to -javaagent:C:\\Users\\sachin\\.m2\\repository\\org\\jacoco\\org.jacoco.agent\\0.8.7\\org.jacoco.agent-0.8.7-runtime.jar=destfile=D:\\java-project\\jhipsterptest-eclipse-workspace\\sample\\multitennnacy\\target\\jacoco.exec -Djava.security.egd=file:/dev/./urandom -Xmx256m
[INFO]
[INFO] --- properties-maven-plugin:1.0.0:read-project-properties (default) # my-app ---
[INFO]
[INFO] --- checksum-maven-plugin:1.10:files (default) # my-app ---
[INFO]
[INFO] --- maven-antrun-plugin:3.0.0:run (eval-frontend-checksum) # my-app ---
[INFO] Executing tasks
[INFO] [copy] Copying 1 file to D:\java-project\jhipsterptest-eclipse-workspace\sample\multitennnacy\target
[INFO] Executed tasks
[INFO]
[INFO] --- frontend-maven-plugin:1.12.0:install-node-and-npm (install-node-and-npm) # my-app ---
[INFO] Node v14.17.1 is already installed.
[INFO] NPM 7.18.1 is already installed.
[INFO]
[INFO] --- frontend-maven-plugin:1.12.0:npm (npm install) # my-app ---
[INFO] Skipping execution.
[INFO]
[INFO] --- frontend-maven-plugin:1.12.0:npm (webapp build dev) # my-app ---
[INFO] Skipping execution.
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) # my-app ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 81 source files to D:\java-project\jhipsterptest-eclipse-workspace\sample\multitennnacy\target\classes
[INFO] /D:/java-project/jhipsterptest-eclipse-workspace/sample/multitennnacy/src/main/java/com/mycompany/myapp/multitenancy/ConnectionProviderFactory.java: D:\java-project\jhipsterptest-eclipse-workspace\sample\multitennnacy\src\main\java\com\mycompany\myapp\multitenancy\ConnectionProviderFactory.java uses unchecked or unsafe operations.
[INFO] /D:/java-project/jhipsterptest-eclipse-workspace/sample/multitennnacy/src/main/java/com/mycompany/myapp/multitenancy/ConnectionProviderFactory.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- maven-resources-plugin:3.2.0:testResources (default-testResources) # my-app ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Using 'UTF-8' encoding to copy filtered properties files.
[INFO] Copying 5 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) # my-app ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 29 source files to D:\java-project\jhipsterptest-eclipse-workspace\sample\multitennnacy\target\test-classes
[INFO] /D:/java-project/jhipsterptest-eclipse-workspace/sample/multitennnacy/src/test/java/com/myapp/service/MailServiceIT.java: D:\java-project\jhipsterptest-eclipse-workspace\sample\multitennnacy\src\test\java\com\myapp\service\MailServiceIT.java uses or overrides a deprecated API.
[INFO] /D:/java-project/jhipsterptest-eclipse-workspace/sample/multitennnacy/src/test/java/com/myapp/service/MailServiceIT.java: Recompile with -Xlint:deprecation for details.
[INFO]
[INFO] <<< spring-boot-maven-plugin:2.4.7:run (default-cli) < test-compile # my-app <<<
[INFO]
[INFO]
[INFO] --- spring-boot-maven-plugin:2.4.7:run (default-cli) # my-app ---
[INFO] Attaching agents: []
??? ??? ??? ????????? ???????? ??????? ????????? ????????? ????????
??? ??? ??? ????????? ????????? ???????? ????????? ????????? ?????????
??? ????????? ??? ????????? ??????? ??? ??????? ?????????
??? ??? ????????? ??? ???????? ??????? ??? ??????? ????????
????????? ??? ??? ????????? ??? ???????? ??? ????????? ??? ????
??????? ??? ??? ????????? ??? ??????? ??? ????????? ??? ???
:: JHipster ? :: Running Spring Boot 2.4.7 ::
:: https://www.jhipster.tech ::
2021-07-14 10:46:05.956 DEBUG 24976 --- [kground-preinit] org.jboss.logging : Logging Provider: org.jboss.logging.Log4j2LoggerProvider
2021-07-14 10:46:05.988 INFO 24976 --- [ restartedMain] com.myapp.MyApp : Starting MyApp using Java 11.0.11 on SACHIN-MSI with PID 24976 (D:\java-project\jhipsterptest-eclipse-workspace\sample\multitennnacy\target\classes started by sachin in D:\java-project\jhipsterptest-eclipse-workspace\sample\multitennnacy)
2021-07-14 10:46:05.989 DEBUG 24976 --- [ restartedMain] com.myapp.MyApp : Running with Spring Boot v2.4.7, Spring v5.3.8
2021-07-14 10:46:05.989 INFO 24976 --- [ restartedMain] com.myapp.MyApp : The following profiles are active: dev,api-docs
2021-07-14 10:46:09.018 DEBUG 24976 --- [ restartedMain] i.m.c.u.i.logging.InternalLoggerFactory : Using SLF4J as the default logging framework
2021-07-14 10:46:09.217 DEBUG 24976 --- [ restartedMain] com.myapp.config.WebConfigurer : Registering CORS filter
2021-07-14 10:46:09.283 INFO 24976 --- [ restartedMain] com.myapp.config.WebConfigurer : Web application configuration, using profiles: dev
2021-07-14 10:46:09.283 INFO 24976 --- [ restartedMain] com.myapp.config.WebConfigurer : Web application fully configured
2021-07-14 10:46:09.570 DEBUG 24976 --- [ restartedMain] c.ehcache.core.Ehcache-usersByLogin : Initialize successful.
2021-07-14 10:46:09.592 DEBUG 24976 --- [ restartedMain] c.ehcache.core.Ehcache-usersByEmail : Initialize successful.
2021-07-14 10:46:09.595 DEBUG 24976 --- [ restartedMain] c.e.core.Ehcache-com.myapp.domain.User : Initialize successful.
2021-07-14 10:46:09.599 DEBUG 24976 --- [ restartedMain] c.e.c.E.myapp.domain.Authority : Initialize successful.
2021-07-14 10:46:09.602 DEBUG 24976 --- [ restartedMain] c.e.c.E.myapp.domain.User.authorities : Initialize successful.
2021-07-14 10:46:09.605 DEBUG 24976 --- [ restartedMain] c.e.c.Ehcache-com.myapp.domain.Tenant : Initialize successful.
2021-07-14 10:46:09.608 DEBUG 24976 --- [ restartedMain] c.e.c.Ehcache-com.myapp.domain.DbType : Initialize successful.
2021-07-14 10:46:09.622 DEBUG 24976 --- [ restartedMain] com.myapp.config.AsyncConfiguration : Creating Async Task Executor
2021-07-14 10:46:09.692 DEBUG 24976 --- [ restartedMain] com.myapp.config.LiquibaseConfiguration : Configuring Liquibase
2021-07-14 10:46:09.829 WARN 24976 --- [ my-app-task-1] t.j.c.liquibase.AsyncSpringLiquibase : Starting Liquibase asynchronously, your database might not be ready at startup!
2021-07-14 10:46:11.299 DEBUG 24976 --- [ restartedMain] com.myapp.security.jwt.TokenProvider : Using a Base64-encoded JWT secret key
2021-07-14 10:46:12.920 DEBUG 24976 --- [ restartedMain] t.j.c.apidoc.SpringfoxAutoConfiguration : Starting OpenAPI docs
2021-07-14 10:46:12.927 DEBUG 24976 --- [ restartedMain] t.j.c.apidoc.SpringfoxAutoConfiguration : Started OpenAPI docs in 5 ms
2021-07-14 10:46:13.623 INFO 24976 --- [ restartedMain] org.jboss.threads : JBoss Threads version 3.1.0.Final
2021-07-14 10:46:14.116 INFO 24976 --- [ restartedMain] com.myapp.MyApp : Started MyApp in 8.783 seconds (JVM running for 9.535)
2021-07-14 10:46:14.124 INFO 24976 --- [ restartedMain] com.myapp.MyApp :
----------------------------------------------------------
Application 'myApp' is running! Access URLs:
Local: http://localhost:8080/
External: http://192.168.43.1:8080/
Profile(s): [dev, api-docs]
----------------------------------------------------------
There are no error logs, but browser shows this.
Seems like I am missing something small, but not ablw to find it out.
I have flow that I want to implement JdbcMetadataStore in the filers, I used SimpleMetadataStore() but this is causing issues as it an in memory store, I need to use a shared metadata store, so I have Postgres DB installed and I can see that Jdbc supports it, I declared a bean as per documents to return a JdbcMetadataStore but I'm not sure how I can use this in the filters, tried to search a lot for any example but could not find a one, note that I'm using FileSystemPersistentAcceptOnceFileListFilter as well as my datasource for Postgres is all setup in my application properties. I have pasted my code here anyone can please guide me on how to move forward?
private DataSource dataSource;
public IntegrationFlow localToFtpFlow(Branch myBranch){
return IntegrationFlows.from(Files.inboundAdapter(new File(myBranch.getBranchCode()))
.filter(new ChainFileListFilter<File>()
.addFilter(new RegexPatternFileListFilter("final" + myBranch.getBranchCode() +".csv"))
.addFilter(new FileSystemPersistentAcceptOnceFileListFilter(new SimpleMetadataStore(), "foo"))),
e -> e.poller(Pollers.fixedDelay(10_000)))
.transform( p ->{
LOG1.info("Sending file " + p + " to FTP branch " + myBranch.getBranchCode());
return p;
})
.log()
.handle(Ftp.outboundAdapter(createNewFtpSessionFactory(myBranch),FileExistsMode.REPLACE)
.useTemporaryFileName(true)
.autoCreateDirectory(false)
.remoteDirectory(myBranch.getFolderPath()))
.get();
}
public DefaultFtpSessionFactory createNewFtpSessionFactory(Branch branch){
final DefaultFtpSessionFactory factory = new DefaultFtpSessionFactory();
factory.setHost(branch.getHost());
factory.setUsername(branch.getUsern());
factory.setPort(branch.getFtpPort());
factory.setPassword(branch.getPassword());
return factory;
}
#Bean
public MetadataStore metadataStore(final DataSource dataSource) {
return new JdbcMetadataStore(dataSource);
}
error arising before I created the table manually, but shouldn't it be created automatically in Postgres as the DB is supported:
org.springframework.messaging.MessagingException: nested exception is org.springframework.jdbc.BadSqlGrammarException: PreparedStatementCallback; bad SQL grammar [INSERT INTO INT_METADATA_STORE(METADATA_KEY, METADATA_VALUE, REGION) SELECT ?, ?, ? FROM INT_METADATA_STORE WHERE METADATA_KEY=? AND REGION=? HAVING COUNT(*)=0]; nested exception is org.postgresql.util.PSQLException: ERROR: relation "int_metadata_store" does not exist
Logging info regarding the issue when adding another flow for the second server, it will trigger the first flow handling method and sending the data to ftp server for both:
Saved Branch : BEY
Hibernate: select branch0_._id as _id1_0_0_, branch0_.branch_code as branch_c2_0_0_, branch0_.folder_path as folder_p3_0_0_, branch0_.ftp_port as ftp_port4_0_0_, branch0_.host as host5_0_0_, branch0_.password as password6_0_0_, branch0_.usern as usern7_0_0_ from branch branch0_ where branch0_._id=?
BEY
2019-01-07 15:11:25.816 INFO 12940 --- [nio-8081-exec-5] o.s.integration.channel.DirectChannel : Channel 'application.intermediateChannel' has 2 subscriber(s).
2019-01-07 15:11:25.817 INFO 12940 --- [nio-8081-exec-5] o.s.i.endpoint.EventDrivenConsumer : started 1.org.springframework.integration.config.ConsumerEndpointFactoryBean#1
2019-01-07 15:11:25.817 INFO 12940 --- [nio-8081-exec-5] o.s.i.endpoint.EventDrivenConsumer : Adding {transformer} as a subscriber to the '1.channel#0' channel
2019-01-07 15:11:25.817 INFO 12940 --- [nio-8081-exec-5] o.s.integration.channel.DirectChannel : Channel 'application.1.channel#0' has 1 subscriber(s).
2019-01-07 15:11:25.817 INFO 12940 --- [nio-8081-exec-5] o.s.i.endpoint.EventDrivenConsumer : started 1.org.springframework.integration.config.ConsumerEndpointFactoryBean#0
2019-01-07 15:11:25.829 INFO 12940 --- [nio-8081-exec-5] o.s.i.e.SourcePollingChannelAdapter : started stockInboundPoller
BEY
2019-01-07 15:11:25.984 INFO 12940 --- [nio-8081-exec-5] o.s.i.endpoint.EventDrivenConsumer : Adding {message-handler} as a subscriber to the '1o.channel#2' channel
2019-01-07 15:11:25.984 INFO 12940 --- [nio-8081-exec-5] o.s.integration.channel.DirectChannel : Channel 'application.1o.channel#2' has 1 subscriber(s).
2019-01-07 15:11:25.984 INFO 12940 --- [nio-8081-exec-5] o.s.i.endpoint.EventDrivenConsumer : started 1o.org.springframework.integration.config.ConsumerEndpointFactoryBean#1
2019-01-07 15:11:25.984 INFO 12940 --- [nio-8081-exec-5] o.s.i.endpoint.EventDrivenConsumer : Adding {transformer} as a subscriber to the '1o.channel#0' channel
2019-01-07 15:11:25.984 INFO 12940 --- [nio-8081-exec-5] o.s.integration.channel.DirectChannel : Channel 'application.1o.channel#0' has 1 subscriber(s).
2019-01-07 15:11:25.984 INFO 12940 --- [nio-8081-exec-5] o.s.i.endpoint.EventDrivenConsumer : started 1o.org.springframework.integration.config.ConsumerEndpointFactoryBean#0
2019-01-07 15:11:25.984 INFO 12940 --- [nio-8081-exec-5] o.s.i.e.SourcePollingChannelAdapter : started 1o.org.springframework.integration.config.SourcePollingChannelAdapterFactoryBean#0
2019-01-07 15:11:42.655 INFO 12940 --- [ask-scheduler-4] o.s.integration.ftp.session.FtpSession : File has been successfully transferred from: /ftp/erbranch/EDMS/FEFO/FEFOexportBEY.csv
Hibernate: select branch0_._id as _id1_0_, branch0_.branch_code as branch_c2_0_, branch0_.folder_path as folder_p3_0_, branch0_.ftp_port as ftp_port4_0_, branch0_.host as host5_0_, branch0_.password as password6_0_, branch0_.usern as usern7_0_ from branch branch0_
Hibernate: insert into branch (branch_code, folder_path, ftp_port, host, password, usern) values (?, ?, ?, ?, ?, ?)
Hibernate: select currval('branch__id_seq')
Saved Branch : JNB
Hibernate: select branch0_._id as _id1_0_0_, branch0_.branch_code as branch_c2_0_0_, branch0_.folder_path as folder_p3_0_0_, branch0_.ftp_port as ftp_port4_0_0_, branch0_.host as host5_0_0_, branch0_.password as password6_0_0_, branch0_.usern as usern7_0_0_ from branch branch0_ where branch0_._id=?
JNB
2019-01-07 15:13:36.099 INFO 12940 --- [nio-8081-exec-7] o.s.integration.channel.DirectChannel : Channel 'application.intermediateChannel' has 3 subscriber(s).
2019-01-07 15:13:36.099 INFO 12940 --- [nio-8081-exec-7] o.s.i.endpoint.EventDrivenConsumer : started 2.org.springframework.integration.config.ConsumerEndpointFactoryBean#1
2019-01-07 15:13:36.099 INFO 12940 --- [nio-8081-exec-7] o.s.i.endpoint.EventDrivenConsumer : Adding {transformer} as a subscriber to the '2.channel#0' channel
2019-01-07 15:13:36.099 INFO 12940 --- [nio-8081-exec-7] o.s.integration.channel.DirectChannel : Channel 'application.2.channel#0' has 1 subscriber(s).
2019-01-07 15:13:36.099 INFO 12940 --- [nio-8081-exec-7] o.s.i.endpoint.EventDrivenConsumer : started 2.org.springframework.integration.config.ConsumerEndpointFactoryBean#0
2019-01-07 15:13:36.099 INFO 12940 --- [nio-8081-exec-7] o.s.i.e.SourcePollingChannelAdapter : started stockInboundPoller
JNB
2019-01-07 15:13:36.130 INFO 12940 --- [nio-8081-exec-7] o.s.i.endpoint.EventDrivenConsumer : Adding {message-handler} as a subscriber to the '2o.channel#2' channel
2019-01-07 15:13:36.135 INFO 12940 --- [nio-8081-exec-7] o.s.integration.channel.DirectChannel : Channel 'application.2o.channel#2' has 1 subscriber(s).
2019-01-07 15:13:36.135 INFO 12940 --- [nio-8081-exec-7] o.s.i.endpoint.EventDrivenConsumer : started 2o.org.springframework.integration.config.ConsumerEndpointFactoryBean#1
2019-01-07 15:13:36.135 INFO 12940 --- [nio-8081-exec-7] o.s.i.endpoint.EventDrivenConsumer : Adding {transformer} as a subscriber to the '2o.channel#0' channel
2019-01-07 15:13:36.135 INFO 12940 --- [nio-8081-exec-7] o.s.integration.channel.DirectChannel : Channel 'application.2o.channel#0' has 1 subscriber(s).
2019-01-07 15:13:36.135 INFO 12940 --- [nio-8081-exec-7] o.s.i.endpoint.EventDrivenConsumer : started 2o.org.springframework.integration.config.ConsumerEndpointFactoryBean#0
2019-01-07 15:13:36.135 INFO 12940 --- [nio-8081-exec-7] o.s.i.e.SourcePollingChannelAdapter : started 2o.org.springframework.integration.config.SourcePollingChannelAdapterFactoryBean#0
2019-01-07 15:13:40.981 INFO 12940 --- [ask-scheduler-1] o.s.integration.ftp.session.FtpSession : File has been successfully transferred from: /ftp/erbranch/EDMS/FEFO/FEFOexportJNB.csv
2019-01-07 15:13:46.085 INFO 12940 --- [ask-scheduler-7] o.s.i.file.FileReadingMessageSource : Created message: [GenericMessage [payload=BEY\finalBEY.csv, headers={file_originalFile=BEY\finalBEY.csv, id=42a97889-7bfb-8f77-75d8-4e7988a368f9, file_name=finalBEY.csv, file_relativePath=finalBEY.csv, timestamp=1546866826085}]]
2019-01-07 15:13:46.086 INFO 12940 --- [ask-scheduler-7] o.s.integration.handler.LoggingHandler : GenericMessage [payload=BEY\finalBEY.csv, headers={file_originalFile=BEY\finalBEY.csv, id=108a92b0-db42-620e-1c46-90652a071220, file_name=finalBEY.csv, file_relativePath=finalBEY.csv, timestamp=1546866826086}]
2019-01-07 15:13:46.160 INFO 12940 --- [ask-scheduler-8] o.s.i.file.FileReadingMessageSource : Created message: [GenericMessage [payload=JNB\finalJNB.csv, headers={file_originalFile=JNB\finalJNB.csv, id=d3b2c6a0-2e9c-42a8-c224-0ed9cbbfaabb, file_name=finalJNB.csv, file_relativePath=finalJNB.csv, timestamp=1546866826160}]]
2019-01-07 15:13:46.161 INFO 12940 --- [ask-scheduler-8] o.s.integration.handler.LoggingHandler : GenericMessage [payload=JNB\finalJNB.csv, headers={file_originalFile=JNB\finalJNB.csv, id=e34070c2-e6ff-e5e1-8c64-4af697ab1032, file_name=finalJNB.csv, file_relativePath=finalJNB.csv, timestamp=1546866826161}]
2019-01-07 15:13:47.129 INFO 12940 --- [ask-scheduler-7] o.s.integration.ftp.session.FtpSession : File has been successfully transferred to: /ftp/erbranch/EDMS/FEFO/finalBEY.csv.writing
2019-01-07 15:13:47.534 INFO 12940 --- [ask-scheduler-7] o.s.integration.ftp.session.FtpSession : File has been successfully renamed from: /ftp/erbranch/EDMS/FEFO/finalBEY.csv.writing to /ftp/erbranch/EDMS/FEFO/finalBEY.csv
2019-01-07 15:13:49.772 INFO 12940 --- [ask-scheduler-8] o.s.integration.ftp.session.FtpSession : File has been successfully transferred to: /ftp/erbranch/EDMS/FEFO/finalJNB.csv.writing
2019-01-07 15:13:50.757 INFO 12940 --- [ask-scheduler-8] o.s.integration.ftp.session.FtpSession : File has been successfully renamed from: /ftp/erbranch/EDMS/FEFO/finalJNB.csv.writing to /ftp/erbranch/EDMS/FEFO/finalJNB.csv
You can find my app here
https://github.com/EliasKhattar/Spring-Integration-Project/tree/master/spring4ftpappftp
Just use metadataSource(dataSource) instead of new SimpleMetadataStore() in the filter constructor.
EDIT
I just copied your flow into a new app (made a few changes, but not to the filter) and everything works fine for me...
#SpringBootApplication
public class So54039852Application {
public static void main(String[] args) {
SpringApplication.run(So54039852Application.class, args);
}
#Bean
public IntegrationFlow localToFtpFlow(DataSource dataSource) {
return IntegrationFlows.from(Files.inboundAdapter(new File("/tmp/foo"))
.filter(new ChainFileListFilter<File>()
.addFilter(new RegexPatternFileListFilter(".*\\.csv"))
.addFilter(new FileSystemPersistentAcceptOnceFileListFilter(metadataStore(dataSource), "foo"))),
e -> e.poller(Pollers.fixedDelay(10_000)))
.log()
.get();
}
#Bean
public ConcurrentMetadataStore metadataStore(final DataSource dataSource) {
return new JdbcMetadataStore(dataSource);
}
}
and
$ touch /tmp/foo/foo.csv
...
$ touch /tmp/foo/bar.csv
and
2019-01-09 12:46:26.332 INFO 43329 --- [ask-scheduler-2] o.s.integration.handler.LoggingHandler : GenericMessage [payload=/tmp/foo/foo.csv, headers={file_originalFile=/tmp/foo/foo.csv, id=e0613529-a657-fbd3-5e67-8bb53a58b5ca, file_name=foo.csv, file_relativePath=foo.csv, timestamp=1547055986330}]
2019-01-09 12:47:26.487 INFO 43329 --- [ask-scheduler-5] o.s.integration.handler.LoggingHandler : GenericMessage [payload=/tmp/foo/bar.csv, headers={file_originalFile=/tmp/foo/bar.csv, id=4feb74b6-d711-f028-70c7-83cdfcd0aeec, file_name=bar.csv, file_relativePath=bar.csv, timestamp=1547056046487}]
and
mysql> select * from INT_METADATA_STORE;
+---------------------+----------------+---------+
| METADATA_KEY | METADATA_VALUE | REGION |
+---------------------+----------------+---------+
| foo/tmp/foo/bar.csv | 1547056039000 | DEFAULT |
| foo/tmp/foo/foo.csv | 1547055980000 | DEFAULT |
+---------------------+----------------+---------+
2 rows in set (0.00 sec)
I don't see the file again even after a restart, but if I change the date on one of the files...
$ touch /tmp/foo/bar.csv
and
2019-01-09 12:51:58.534 INFO 44430 --- [ask-scheduler-2] o.s.integration.handler.LoggingHandler : GenericMessage [payload=/tmp/foo/bar.csv, headers={file_originalFile=/tmp/foo/bar.csv, id=f92d6b36-c948-37cc-ca56-9ef28de336f2, file_name=bar.csv, file_relativePath=bar.csv, timestamp=1547056318532}]
Can I have a job with just the slaves and no master and listen to a rabbitmq queue? I want to listen to a queue and process the messages in chunk oriented manner using spring batch and spring integration in a spring boot app.
I want to use the chunkProcessorChunkHandler configuration explained in the RemoteChunking example for Spring batch by Michael Minella (https://www.youtube.com/watch?v=30Tdp1mfR0g), but without a master configuration.
Below is my configuration for the job.
#Configuration
#EnableIntegration
public class QueueIntegrationConfiguration {
#Autowired
private CassandraItemWriter cassandraItemWriter;
#Autowired
private VendorProcessor vendorProcessor;
#Autowired
ConnectionFactory connectionFactory;
#Bean
public AmqpInboundChannelAdapter inboundChannelAdapter(
SimpleMessageListenerContainer listenerContainer) {
AmqpInboundChannelAdapter adapter = new AmqpInboundChannelAdapter(listenerContainer);
adapter.setOutputChannel(inboundQueueChannel());
adapter.setAutoStartup(true);
return adapter;
}
#Bean
public SimpleMessageListenerContainer listenerContainer(ConnectionFactory connectionFactory, MessageConverter jsonMessageConverter) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer(
connectionFactory);
container.setQueueNames("ProductStore_Partial");
container.setAutoStartup(true);
container.setMessageConverter(jsonMessageConverter);
return container;
}
#Bean
#ServiceActivator(inputChannel = "ProductStore_Partial")
public ChunkProcessorChunkHandler chunkProcessorChunkHandler()
throws Exception {
SimpleChunkProcessor chunkProcessor = new SimpleChunkProcessor(vendorProcessor,
cassandraItemWriter);
chunkProcessor.afterPropertiesSet();
ChunkProcessorChunkHandler<Vendor> chunkHandler = new ChunkProcessorChunkHandler<>();
chunkHandler.setChunkProcessor(chunkProcessor);
chunkHandler.afterPropertiesSet();
return chunkHandler;
}
#Bean
public QueueChannel inboundQueueChannel() {
return new QueueChannel().;
}
}
Below is my Application.java class for spring boot.
#SpringBootApplication
#EnableBatchProcessing
public class BulkImportProductApplication {
public static void main(String[] args) {
SpringApplication app = new SpringApplication(BulkImportProductApplication.class);
app.setWebEnvironment(false);
app.run(args).close();
}
}
From what I understand from spring integration, I have an AmqpInboundChannelAdapter for listening to messages from the queue. A ServiceActivator, an inboundQueueChannel, autowired ItemProcessor and ItemWriter. I am not sure what am I missing here.
The batch job starts, consumes one message from the queue and get a cancelOk and my job terminates without processing the message.
I am also sharing my debug logging if that would help.
2017-12-04 09:58:49.679 INFO 7450 --- [ main] c.a.s.p.b.BulkImportProductApplication : Started BulkImportProductApplication in 9.412 seconds (JVM running for 10.39)
2017-12-04 09:58:49.679 INFO 7450 --- [ main] s.c.a.AnnotationConfigApplicationContext : Closing org.springframework.context.annotation.AnnotationConfigApplicationContext#31c88ec8: startup date [Mon Dec 04 09:58:40 PST 2017]; root of context hierarchy
2017-12-04 09:58:49.679 DEBUG 7450 --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'org.springframework.integration.config.IdGeneratorConfigurer#0'
2017-12-04 09:58:49.680 DEBUG 7450 --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'inboundChannelAdapter'
2017-12-04 09:58:49.680 DEBUG 7450 --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'listenerContainer'
2017-12-04 09:58:49.680 DEBUG 7450 --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'integrationHeaderChannelRegistry'
2017-12-04 09:58:49.680 DEBUG 7450 --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'org.springframework.amqp.rabbit.config.internalRabbitListenerEndpointRegistry'
2017-12-04 09:58:49.680 DEBUG 7450 --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean '_org.springframework.integration.errorLogger'
2017-12-04 09:58:49.680 DEBUG 7450 --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'queueIntegrationConfiguration.chunkProcessorChunkHandler.serviceActivator.handler'
2017-12-04 09:58:49.680 DEBUG 7450 --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'queueIntegrationConfiguration.chunkProcessorChunkHandler.serviceActivator'
2017-12-04 09:58:49.680 DEBUG 7450 --- [ main] o.s.b.f.s.DefaultListableBeanFactory : Returning cached instance of singleton bean 'lifecycleProcessor'
2017-12-04 09:58:49.680 INFO 7450 --- [ main] o.s.c.support.DefaultLifecycleProcessor : Stopping beans in phase 2147483647
2017-12-04 09:58:49.680 DEBUG 7450 --- [ main] o.s.c.support.DefaultLifecycleProcessor : Asking bean 'inboundChannelAdapter' of type [class org.springframework.integration.amqp.inbound.AmqpInboundChannelAdapter] to stop
2017-12-04 09:58:49.680 DEBUG 7450 --- [ main] o.s.a.r.l.SimpleMessageListenerContainer : Shutting down Rabbit listener container
2017-12-04 09:58:49.814 DEBUG 7450 --- [pool-1-thread-5] o.s.a.r.listener.BlockingQueueConsumer : Storing delivery for Consumer#7c52fc81: tags=[{}], channel=Cached Rabbit Channel: AMQChannel(amqp://admin#xxxx:5672/,2), conn: Proxy#26f1249d Shared Rabbit Connection: SimpleConnection#680bddf5 [delegate=amqp://admin#xxxx:5672/, localPort= 65035], acknowledgeMode=AUTO local queue size=0
2017-12-04 09:58:49.814 DEBUG 7450 --- [enerContainer-1] o.s.a.r.listener.BlockingQueueConsumer : Received message: (Body:'[B#358a5358(byte[618])' MessageProperties [headers={__TypeId__=com.art.service.product.bulkimportproduct.data.model.Vendor}, timestamp=null, messageId=null, userId=null, receivedUserId=null, appId=null, clusterId=null, type=null, correlationId=null, correlationIdString=null, replyTo=null, contentType=json, contentEncoding=UTF-8, contentLength=0, deliveryMode=null, receivedDeliveryMode=NON_PERSISTENT, expiration=null, priority=0, redelivered=false, receivedExchange=ProductStore, receivedRoutingKey=, receivedDelay=null, deliveryTag=2, messageCount=0, consumerTag=amq.ctag-nWGbRxjFiaeTEoZylv6Hrg, consumerQueue=null])
2017-12-04 09:58:49.815 DEBUG 7450 --- [enerContainer-1] s.i.m.AbstractHeaderMapper$HeaderMatcher : headerName=[amqp_receivedDeliveryMode] WILL be mapped, matched pattern=*
2017-12-04 09:58:49.815 DEBUG 7450 --- [enerContainer-1] s.i.m.AbstractHeaderMapper$HeaderMatcher : headerName=[amqp_contentEncoding] WILL be mapped, matched pattern=*
2017-12-04 09:58:49.815 DEBUG 7450 --- [enerContainer-1] s.i.m.AbstractHeaderMapper$HeaderMatcher : headerName=[amqp_receivedExchange] WILL be mapped, matched pattern=*
2017-12-04 09:58:49.815 DEBUG 7450 --- [enerContainer-1] s.i.m.AbstractHeaderMapper$HeaderMatcher : headerName=[amqp_deliveryTag] WILL be mapped, matched pattern=*
2017-12-04 09:58:49.815 DEBUG 7450 --- [enerContainer-1] s.i.m.AbstractHeaderMapper$HeaderMatcher : headerName=[json__TypeId__] WILL be mapped, matched pattern=*
2017-12-04 09:58:49.815 DEBUG 7450 --- [enerContainer-1] s.i.m.AbstractHeaderMapper$HeaderMatcher : headerName=[amqp_redelivered] WILL be mapped, matched pattern=*
2017-12-04 09:58:49.815 DEBUG 7450 --- [enerContainer-1] s.i.m.AbstractHeaderMapper$HeaderMatcher : headerName=[contentType] WILL be mapped, matched pattern=*
2017-12-04 09:58:49.815 DEBUG 7450 --- [enerContainer-1] s.i.m.AbstractHeaderMapper$HeaderMatcher : headerName=[__TypeId__] WILL be mapped, matched pattern=*
2017-12-04 09:58:49.815 DEBUG 7450 --- [enerContainer-1] o.s.integration.channel.QueueChannel : preSend on channel 'inboundQueueChannel', message: GenericMessage [payload=byte[618], headers={amqp_receivedDeliveryMode=NON_PERSISTENT, amqp_contentEncoding=UTF-8, amqp_receivedExchange=ProductStore, amqp_deliveryTag=2, json__TypeId__=com.art.service.product.bulkimportproduct.data.model.Vendor, amqp_redelivered=false, id=a4868670-240f-ddf2-8a8c-ac4b8d234cdd, amqp_consumerTag=amq.ctag-nWGbRxjFiaeTEoZylv6Hrg, contentType=json, __TypeId__=com.art.service.product.bulkimportproduct.data.model.Vendor, timestamp=1512410329815}]
2017-12-04 09:58:49.815 DEBUG 7450 --- [enerContainer-1] o.s.integration.channel.QueueChannel : postSend (sent=true) on channel 'inboundQueueChannel', message: GenericMessage [payload=byte[618], headers={amqp_receivedDeliveryMode=NON_PERSISTENT, amqp_contentEncoding=UTF-8, amqp_receivedExchange=ProductStore, amqp_deliveryTag=2, json__TypeId__=com.art.service.product.bulkimportproduct.data.model.Vendor, amqp_redelivered=false, id=a4868670-240f-ddf2-8a8c-ac4b8d234cdd, amqp_consumerTag=amq.ctag-nWGbRxjFiaeTEoZylv6Hrg, contentType=json, __TypeId__=com.art.service.product.bulkimportproduct.data.model.Vendor, timestamp=1512410329815}]
2017-12-04 09:58:49.853 INFO 7450 --- [ main] o.s.a.r.l.SimpleMessageListenerContainer : Waiting for workers to finish.
2017-12-04 09:58:49.853 DEBUG 7450 --- [pool-1-thread-6] o.s.a.r.listener.BlockingQueueConsumer : Received cancelOk for tag amq.ctag-nWGbRxjFiaeTEoZylv6Hrg (null); Consumer#7c52fc81: tags=[{}], channel=Cached Rabbit Channel: AMQChannel(amqp://admin#xxxx:5672/,2), conn: Proxy#26f1249d Shared Rabbit Connection: SimpleConnection#680bddf5 [delegate=amqp://admin#xxxx:5672/, localPort= 65035], acknowledgeMode=AUTO local queue size=0
2017-12-04 09:58:49.853 DEBUG 7450 --- [enerContainer-1] o.s.a.r.l.SimpleMessageListenerContainer : Cancelling Consumer#7c52fc81: tags=[{}], channel=Cached Rabbit Channel: AMQChannel(amqp://admin#xxxx:5672/,2), conn: Proxy#26f1249d Shared Rabbit Connection: SimpleConnection#680bddf5 [delegate=amqp://admin#xxxx:5672/, localPort= 65035], acknowledgeMode=AUTO local queue size=0
2017-12-04 09:58:49.853 DEBUG 7450 --- [enerContainer-1] o.s.a.r.listener.BlockingQueueConsumer : Closing Rabbit Channel: Cached Rabbit Channel: AMQChannel(amqp://admin#xxxx:5672/,2), conn: Proxy#26f1249d Shared Rabbit Connection: SimpleConnection#680bddf5 [delegate=amqp://admin#xxxx:5672/, localPort= 65035]
2017-12-04 09:58:49.853 DEBUG 7450 --- [enerContainer-1] o.s.a.r.c.CachingConnectionFactory : Closing cached Channel: AMQChannel(amqp://admin#xxxx:5672/,2)
2017-12-04 09:58:50.027 INFO 7450 --- [ main] o.s.a.r.l.SimpleMessageListenerContainer : Successfully waited for workers to finish.
2017-12-04 09:58:50.027 DEBUG 7450 --- [ main] o.s.c.support.DefaultLifecycleProcessor : Bean 'inboundChannelAdapter' completed its stop procedure
What am I missing here? Why is my message getting processed? Please correct me if I'm missing out something here? Also feel free to ask any other configuration that you feel would help analyze the situation here.
EDIT: After removing the code that closes the application context manually( app.run(args).close() ), I was able to receive the messages, but looks like they are lost after a successful retrieve. sharing the debug log for this behavior.
2017-12-04 14:39:11.297 DEBUG 1498 --- [pool-1-thread-5] o.s.a.r.listener.BlockingQueueConsumer : Storing delivery for Consumer#7219ac49: tags=[{amq.ctag-Z8siptJMdxGU6sXdOHkVCA=ProductStore_Partial}], channel=Cached Rabbit Channel: AMQChannel(amqp://admin#xxxx:5672/,2), conn: Proxy#6df20ade Shared Rabbit Connection: SimpleConnection#7ba63fe5 [delegate=amqp://admin#xxxx:5672/, localPort= 51172], acknowledgeMode=AUTO local queue size=0
2017-12-04 14:39:11.297 DEBUG 1498 --- [enerContainer-1] o.s.a.r.listener.BlockingQueueConsumer : Received message: (Body:'[B#347c8f87(byte[624])' MessageProperties [headers={__TypeId__=com.art.service.product.bulkimportproduct.data.model.Vendor}, timestamp=null, messageId=null, userId=null, receivedUserId=null, appId=null, clusterId=null, type=null, correlationId=null, correlationIdString=null, replyTo=null, contentType=json, contentEncoding=UTF-8, contentLength=0, deliveryMode=null, receivedDeliveryMode=NON_PERSISTENT, expiration=null, priority=0, redelivered=false, receivedExchange=ProductStore, receivedRoutingKey=, receivedDelay=null, deliveryTag=2, messageCount=0, consumerTag=amq.ctag-Z8siptJMdxGU6sXdOHkVCA, consumerQueue=ProductStore_Partial])
2017-12-04 14:39:11.297 DEBUG 1498 --- [enerContainer-1] s.i.m.AbstractHeaderMapper$HeaderMatcher : headerName=[amqp_receivedDeliveryMode] WILL be mapped, matched pattern=*
2017-12-04 14:39:11.297 DEBUG 1498 --- [enerContainer-1] s.i.m.AbstractHeaderMapper$HeaderMatcher : headerName=[amqp_contentEncoding] WILL be mapped, matched pattern=*
2017-12-04 14:39:11.297 DEBUG 1498 --- [enerContainer-1] s.i.m.AbstractHeaderMapper$HeaderMatcher : headerName=[amqp_receivedExchange] WILL be mapped, matched pattern=*
2017-12-04 14:39:11.297 DEBUG 1498 --- [enerContainer-1] s.i.m.AbstractHeaderMapper$HeaderMatcher : headerName=[amqp_deliveryTag] WILL be mapped, matched pattern=*
2017-12-04 14:39:11.297 DEBUG 1498 --- [enerContainer-1] s.i.m.AbstractHeaderMapper$HeaderMatcher : headerName=[json__TypeId__] WILL be mapped, matched pattern=*
2017-12-04 14:39:11.297 DEBUG 1498 --- [enerContainer-1] s.i.m.AbstractHeaderMapper$HeaderMatcher : headerName=[amqp_redelivered] WILL be mapped, matched pattern=*
2017-12-04 14:39:11.297 DEBUG 1498 --- [enerContainer-1] s.i.m.AbstractHeaderMapper$HeaderMatcher : headerName=[contentType] WILL be mapped, matched pattern=*
2017-12-04 14:39:11.297 DEBUG 1498 --- [enerContainer-1] s.i.m.AbstractHeaderMapper$HeaderMatcher : headerName=[__TypeId__] WILL be mapped, matched pattern=*
2017-12-04 14:39:11.297 DEBUG 1498 --- [enerContainer-1] o.s.integration.channel.QueueChannel : preSend on channel 'inboundQueueChannel', message: GenericMessage [payload=byte[624], headers={amqp_receivedDeliveryMode=NON_PERSISTENT, amqp_contentEncoding=UTF-8, amqp_receivedExchange=ProductStore, amqp_deliveryTag=2, json__TypeId__=com.art.service.product.bulkimportproduct.data.model.Vendor, amqp_consumerQueue=ProductStore_Partial, amqp_redelivered=false, id=540399a5-62a6-7178-2524-e274bad4ed13, amqp_consumerTag=amq.ctag-Z8siptJMdxGU6sXdOHkVCA, contentType=json, __TypeId__=com.art.service.product.bulkimportproduct.data.model.Vendor, timestamp=1512427151297}]
2017-12-04 14:39:11.297 DEBUG 1498 --- [enerContainer-1] o.s.integration.channel.QueueChannel : postSend (sent=true) on channel 'inboundQueueChannel', message: GenericMessage [payload=byte[624], headers={amqp_receivedDeliveryMode=NON_PERSISTENT, amqp_contentEncoding=UTF-8, amqp_receivedExchange=ProductStore, amqp_deliveryTag=2, json__TypeId__=com.art.service.product.bulkimportproduct.data.model.Vendor, amqp_consumerQueue=ProductStore_Partial, amqp_redelivered=false, id=540399a5-62a6-7178-2524-e274bad4ed13, amqp_consumerTag=amq.ctag-Z8siptJMdxGU6sXdOHkVCA, contentType=json, __TypeId__=com.art.service.product.bulkimportproduct.data.model.Vendor, timestamp=1512427151297}]
2017-12-04 14:39:11.297 DEBUG 1498 --- [enerContainer-1] o.s.a.r.listener.BlockingQueueConsumer : Retrieving delivery for Consumer#7219ac49: tags=[{amq.ctag-Z8siptJMdxGU6sXdOHkVCA=ProductStore_Partial}], channel=Cached Rabbit Channel: AMQChannel(amqp://admin#xxxx:5672/,2), conn: Proxy#6df20ade Shared Rabbit Connection: SimpleConnection#7ba63fe5 [delegate=amqp://admin#xxxx:5672/, localPort= 51172], acknowledgeMode=AUTO local queue size=0
This goes on repeating and new messages are consumed, but the messages are not getting processed and written to the data-store using the itemWriter provided. Now come to think of it, since I have not provided the tasklet/step bean reference anywhere in this code, is that something I am missing out here?
app.run(args).close();
You are explicitly closing the application context, which shuts everything down.
Closing org.springframework.context.annotation.AnnotationConfigApplicationContext.
I'm new in jhipster, I cloned the sample generated Angular 4 application :
https://github.com/jhipster/jhipster-sample-app-ng2.git
I run the project with eclipse, but the application is not working in the browser !!!
I did't make any change in the source, this is my console :
The Class-Path manifest attribute in C:\Users\HATIME\.m2\repository\org\liquibase\liquibase-core\3.5.3\liquibase-core-3.5.3.jar referenced one or more files that do not exist: C:\Users\HATIME\.m2\repository\org\liquibase\liquibase-core\3.5.3\lib\snakeyaml-1.13.jar
20:25:21.854 [main] DEBUG org.springframework.boot.devtools.settings.DevToolsSettings - Included patterns for restart : []
20:25:21.860 [main] DEBUG org.springframework.boot.devtools.settings.DevToolsSettings - Excluded patterns for restart : [/spring-boot-starter/target/classes/, /spring-boot-autoconfigure/target/classes/, /spring-boot-starter-[\w-]+/, /spring-boot/target/classes/, /spring-boot-actuator/target/classes/, /spring-boot-devtools/target/classes/]
20:25:21.861 [main] DEBUG org.springframework.boot.devtools.restart.ChangeableUrls - Matching URLs for reloading : [file:/D:/Work/workspace/jhipster-sample-app-ng2/target/classes/]
██╗ ██╗ ██╗ ████████╗ ███████╗ ██████╗ ████████╗ ████████╗ ███████╗
██║ ██║ ██║ ╚══██╔══╝ ██╔═══██╗ ██╔════╝ ╚══██╔══╝ ██╔═════╝ ██╔═══██╗
██║ ████████║ ██║ ███████╔╝ ╚█████╗ ██║ ██████╗ ███████╔╝
██╗ ██║ ██╔═══██║ ██║ ██╔════╝ ╚═══██╗ ██║ ██╔═══╝ ██╔══██║
╚██████╔╝ ██║ ██║ ████████╗ ██║ ██████╔╝ ██║ ████████╗ ██║ ╚██╗
╚═════╝ ╚═╝ ╚═╝ ╚═══════╝ ╚═╝ ╚═════╝ ╚═╝ ╚═══════╝ ╚═╝ ╚═╝
:: JHipster 🤓 :: Running Spring Boot 1.5.7.RELEASE ::
:: http://www.jhipster.tech ::
2017-10-26 20:25:25.190 INFO 4916 --- [ restartedMain] i.g.j.s.JhipsterSampleApplicationNg2App : Starting JhipsterSampleApplicationNg2App on HATIME-PC with PID 4916 (D:\Work\workspace\jhipster-sample-app-ng2\target\classes started by HATIME in D:\Work\workspace\jhipster-sample-app-ng2)
2017-10-26 20:25:25.192 DEBUG 4916 --- [ restartedMain] i.g.j.s.JhipsterSampleApplicationNg2App : Running with Spring Boot v1.5.7.RELEASE, Spring v4.3.11.RELEASE
2017-10-26 20:25:25.193 INFO 4916 --- [ restartedMain] i.g.j.s.JhipsterSampleApplicationNg2App : The following profiles are active: swagger,dev
2017-10-26 20:25:25.620 DEBUG 4916 --- [kground-preinit] org.jboss.logging : Logging Provider: org.jboss.logging.Slf4jLoggerProvider found via system property
2017-10-26 20:25:31.188 DEBUG 4916 --- [ restartedMain] i.g.j.sample.config.AsyncConfiguration : Creating Async Task Executor
2017-10-26 20:25:32.757 DEBUG 4916 --- [ restartedMain] class org.ehcache.core.Ehcache-users : Initialize successful.
2017-10-26 20:25:32.792 DEBUG 4916 --- [ restartedMain] c.e.c.E.g.jhipster.sample.domain.User : Initialize successful.
2017-10-26 20:25:32.800 DEBUG 4916 --- [ restartedMain] c.e.c.E.g.j.sample.domain.Authority : Initialize successful.
2017-10-26 20:25:32.805 DEBUG 4916 --- [ restartedMain] c.e.c.E.g.j.s.domain.User.authorities : Initialize successful.
2017-10-26 20:25:32.811 DEBUG 4916 --- [ restartedMain] c.e.c.E.g.j.sample.domain.BankAccount : Initialize successful.
2017-10-26 20:25:32.816 DEBUG 4916 --- [ restartedMain] c.e.c.E.g.j.s.d.BankAccount.operations : Initialize successful.
2017-10-26 20:25:32.822 DEBUG 4916 --- [ restartedMain] c.e.c.E.g.jhipster.sample.domain.Label : Initialize successful.
2017-10-26 20:25:32.831 DEBUG 4916 --- [ restartedMain] c.e.c.E.g.j.s.domain.Label.operations : Initialize successful.
2017-10-26 20:25:32.836 DEBUG 4916 --- [ restartedMain] c.e.c.E.g.j.sample.domain.Operation : Initialize successful.
2017-10-26 20:25:32.844 DEBUG 4916 --- [ restartedMain] c.e.c.E.g.j.s.domain.Operation.labels : Initialize successful.
2017-10-26 20:25:33.326 DEBUG 4916 --- [ restartedMain] i.g.j.s.config.MetricsConfiguration : Registering JVM gauges
2017-10-26 20:25:33.360 DEBUG 4916 --- [ restartedMain] i.g.j.s.config.MetricsConfiguration : Monitoring the datasource
2017-10-26 20:25:33.361 DEBUG 4916 --- [ restartedMain] i.g.j.s.config.MetricsConfiguration : Initializing Metrics JMX reporting
2017-10-26 20:25:35.101 DEBUG 4916 --- [ restartedMain] i.g.j.sample.config.WebConfigurer : Registering CORS filter
2017-10-26 20:25:35.421 INFO 4916 --- [ restartedMain] i.g.j.sample.config.WebConfigurer : Web application configuration, using profiles: swagger
2017-10-26 20:25:35.422 DEBUG 4916 --- [ restartedMain] i.g.j.sample.config.WebConfigurer : Initializing Metrics registries
2017-10-26 20:25:35.428 DEBUG 4916 --- [ restartedMain] i.g.j.sample.config.WebConfigurer : Registering Metrics Filter
2017-10-26 20:25:35.429 DEBUG 4916 --- [ restartedMain] i.g.j.sample.config.WebConfigurer : Registering Metrics Servlet
2017-10-26 20:25:35.435 DEBUG 4916 --- [ restartedMain] i.g.j.sample.config.WebConfigurer : Initialize H2 console
2017-10-26 20:25:35.436 INFO 4916 --- [ restartedMain] i.g.j.sample.config.WebConfigurer : Web application fully configured
2017-10-26 20:25:36.046 DEBUG 4916 --- [ restartedMain] i.g.j.s.config.DatabaseConfiguration : Configuring Liquibase
2017-10-26 20:25:36.071 WARN 4916 --- [ng-2-Executor-1] i.g.j.c.liquibase.AsyncSpringLiquibase : Starting Liquibase asynchronously, your database might not be ready at startup!
2017-10-26 20:25:38.669 DEBUG 4916 --- [ng-2-Executor-1] i.g.j.c.liquibase.AsyncSpringLiquibase : Liquibase has updated your database in 2596 ms
2017-10-26 20:25:51.678 DEBUG 4916 --- [ restartedMain] i.g.j.c.apidoc.SwaggerConfiguration : Starting Swagger
2017-10-26 20:25:51.690 DEBUG 4916 --- [ restartedMain] i.g.j.c.apidoc.SwaggerConfiguration : Started Swagger in 12 ms
2017-10-26 20:25:54.999 INFO 4916 --- [ restartedMain] i.g.j.s.JhipsterSampleApplicationNg2App : Started JhipsterSampleApplicationNg2App in 33.101 seconds (JVM running for 34.175)
2017-10-26 20:25:55.000 INFO 4916 --- [ restartedMain] i.g.j.s.JhipsterSampleApplicationNg2App :
----------------------------------------------------------
Application 'jhipsterSampleApplicationNG2' is running! Access URLs:
Local: http://localhost:8080
External: http://*.*.*.*:8080
Profile(s): [swagger, dev]
----------------------------------------------------------
Did you launch yarn install to get the node_modules folder, and compile the front ?
Try:
yarn install
yarn run webpack:build
yarn start
My jhipster-registry stopped working a couple of hours ago.
I followd these steps:
removed node_modules/ folder
npm install
./mvnw
When the registry is up and running again I get an error page with the following text:
??error.title_en??
And in the logs find:
2017-08-04 22:58:53.729 DEBUG 3203 --- [kground-preinit] org.jboss.logging : Logging Provider: org.jboss.logging.Slf4jLoggerProvider found via system property
██╗ ██╗ ██╗ ████████╗ ███████╗ ██████╗ ████████╗ ████████╗ ███████╗
██║ ██║ ██║ ╚══██╔══╝ ██╔═══██╗ ██╔════╝ ╚══██╔══╝ ██╔═════╝ ██╔═══██╗
██║ ████████║ ██║ ███████╔╝ ╚█████╗ ██║ ██████╗ ███████╔╝
██╗ ██║ ██╔═══██║ ██║ ██╔════╝ ╚═══██╗ ██║ ██╔═══╝ ██╔══██║
╚██████╔╝ ██║ ██║ ████████╗ ██║ ██████╔╝ ██║ ████████╗ ██║ ╚██╗
╚═════╝ ╚═╝ ╚═╝ ╚═══════╝ ╚═╝ ╚═════╝ ╚═╝ ╚═══════╝ ╚═╝ ╚═╝
:: JHipster Registry :: Running Spring Boot 1.5.4.RELEASE ::
:: http://jhipster.github.io ::
2017-08-04 22:58:56.117 INFO 3203 --- [ restartedMain] i.g.j.registry.JHipsterRegistryApp : The following profiles are active: dev,native
2017-08-04 22:58:59.313 DEBUG 3203 --- [ restartedMain] i.g.j.r.config.AsyncConfiguration : Creating Async Task Executor
2017-08-04 22:58:59.994 DEBUG 3203 --- [ restartedMain] i.g.j.r.config.MetricsConfiguration : Registering JVM gauges
2017-08-04 22:59:00.274 DEBUG 3203 --- [ restartedMain] i.g.j.r.config.MetricsConfiguration : Initializing Metrics JMX reporting
2017-08-04 22:59:03.135 DEBUG 3203 --- [ restartedMain] i.g.j.registry.config.WebConfigurer : Registering CORS filter
2017-08-04 22:59:03.327 WARN 3203 --- [ restartedMain] c.n.c.sources.URLConfigurationSource : No URLs will be polled as dynamic configuration sources.
2017-08-04 22:59:04.102 INFO 3203 --- [ restartedMain] i.g.j.registry.config.WebConfigurer : Web application configuration, using profiles: dev
2017-08-04 22:59:04.103 DEBUG 3203 --- [ restartedMain] i.g.j.registry.config.WebConfigurer : Initializing Metrics registries
2017-08-04 22:59:04.107 DEBUG 3203 --- [ restartedMain] i.g.j.registry.config.WebConfigurer : Registering Metrics Filter
2017-08-04 22:59:04.107 DEBUG 3203 --- [ restartedMain] i.g.j.registry.config.WebConfigurer : Registering Metrics Servlet
2017-08-04 22:59:04.113 INFO 3203 --- [ restartedMain] i.g.j.registry.config.WebConfigurer : Web application fully configured
2017-08-04 22:59:12.109 WARN 3203 --- [ restartedMain] c.n.c.sources.URLConfigurationSource : No URLs will be polled as dynamic configuration sources.
2017-08-04 22:59:12.551 INFO 3203 --- [ restartedMain] com.netflix.discovery.DiscoveryClient : Initializing Eureka in region us-east-1
2017-08-04 22:59:12.551 INFO 3203 --- [ restartedMain] com.netflix.discovery.DiscoveryClient : Client configured to neither register nor query for data.
2017-08-04 22:59:12.571 INFO 3203 --- [ restartedMain] com.netflix.discovery.DiscoveryClient : Discovery Client initialized at timestamp 1501880352571 with initial instances count: 0
2017-08-04 22:59:12.941 INFO 3203 --- [ restartedMain] c.n.d.provider.DiscoveryJerseyProvider : Using JSON encoding codec LegacyJacksonJson
2017-08-04 22:59:12.941 INFO 3203 --- [ restartedMain] c.n.d.provider.DiscoveryJerseyProvider : Using JSON decoding codec LegacyJacksonJson
2017-08-04 22:59:12.942 INFO 3203 --- [ restartedMain] c.n.d.provider.DiscoveryJerseyProvider : Using XML encoding codec XStreamXml
2017-08-04 22:59:12.942 INFO 3203 --- [ restartedMain] c.n.d.provider.DiscoveryJerseyProvider : Using XML decoding codec XStreamXml
2017-08-04 22:59:13.809 WARN 3203 --- [ restartedMain] o.s.j.e.a.AnnotationMBeanExporter : Bean with key 'zuulEndpoint' has been registered as an MBean but has no exposed attributes or operations
2017-08-04 22:59:14.606 DEBUG 3203 --- [pool-4-thread-1] i.g.j.r.aop.logging.LoggingAspect : Enter: io.github.jhipster.registry.service.ZuulUpdaterService.updateZuulRoutes() with argument[s] = []
2017-08-04 22:59:14.614 DEBUG 3203 --- [pool-4-thread-1] i.g.j.r.aop.logging.LoggingAspect : Exit: io.github.jhipster.registry.service.ZuulUpdaterService.updateZuulRoutes() with result = null
2017-08-04 22:59:14.740 INFO 3203 --- [ restartedMain] i.g.j.registry.JHipsterRegistryApp : Started JHipsterRegistryApp in 22.799 seconds (JVM running for 23.886)
2017-08-04 22:59:14.741 INFO 3203 --- [ restartedMain] i.g.j.registry.JHipsterRegistryApp :
----------------------------------------------------------
Application 'jhipster-registry' is running! Access URLs:
Local: http://localhost:8761
External: http://xxx.xxx.x.xxx:8761
Profile(s): [dev, native]
----------------------------------------------------------
2017-08-04 22:59:19.615 DEBUG 3203 --- [pool-4-thread-1] i.g.j.r.aop.logging.LoggingAspect : Enter: io.github.jhipster.registry.service.ZuulUpdaterService.updateZuulRoutes() with argument[s] = []
2017-08-04 22:59:19.616 DEBUG 3203 --- [pool-4-thread-1] i.g.j.r.aop.logging.LoggingAspect : Exit: io.github.jhipster.registry.service.ZuulUpdaterService.updateZuulRoutes() with result = null
2017-08-04 22:59:24.621 DEBUG 3203 --- [pool-4-thread-1] i.g.j.r.aop.logging.LoggingAspect : Enter: io.github.jhipster.registry.service.ZuulUpdaterService.updateZuulRoutes() with argument[s] = []
2017-08-04 22:59:24.622 DEBUG 3203 --- [pool-4-thread-1] i.g.j.r.aop.logging.LoggingAspect : Exit: io.github.jhipster.registry.service.ZuulUpdaterService.updateZuulRoutes() with result = null
2017-08-04 22:59:27.234 WARN 3203 --- [ XNIO-2 task-1] o.s.c.n.zuul.web.ZuulHandlerMapping : No routes found from RouteLocator
2017-08-04 22:59:27.438 DEBUG 3203 --- [ XNIO-2 task-1] freemarker.cache : Couldn't find template in cache for "error.ftl"("en", UTF-8, parsed); will try to load it.
2017-08-04 22:59:27.441 DEBUG 3203 --- [ XNIO-2 task-1] freemarker.cache : TemplateLoader.findTemplateSource("error_en.ftl"): Not found
2017-08-04 22:59:27.444 DEBUG 3203 --- [ XNIO-2 task-1] freemarker.cache : TemplateLoader.findTemplateSource("error.ftl"): Not found
2017-08-04 22:59:29.623 DEBUG 3203 --- [pool-4-thread-1] i.g.j.r.aop.logging.LoggingAspect : Enter: io.github.jhipster.registry.service.ZuulUpdaterService.updateZuulRoutes() with argument[s] = []
2017-08-04 22:59:29.623 DEBUG 3203 --- [pool-4-thread-1] i.g.j.r.aop.logging.LoggingAspect : Exit: io.github.jhipster.registry.service.ZuulUpdaterService.updateZuulRoutes()
I don't know where all this come from.
Edit-----------------------------------
The problem may be when I run npm install and it's postinstall script "postinstall": "yarn run webpack:build" an ERROR is thrown:
ERROR in ./~/css-loader!./~/postcss-loader!./~/sass-loader/lib/loader.js!./src/main/webapp/content/scss/vendor.scss
Module build failed: TypeError: Invalid PostCSS Plugin found: [0]
at /Users/xxxxxxx/dev/projects/registrytest/node_modules/postcss-load-plugins/lib/plugins.js:32:17
at Array.forEach (native)
at plugins (/Users/xxxxxxx/dev/projects/registrytest/node_modules/postcss-load-plugins/lib/plugins.js:21:15)
at /Users/xxxxxxx/dev/projects/registrytest/node_modules/postcss-load-config/index.js:64:18
at <anonymous>
# ./src/main/webapp/content/scss/vendor.scss 4:14-194
# ./src/main/webapp/app/vendor.ts
# dll vendor
error Command failed with exit code 2.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! jhipster-registry#3.1.0 postinstall: `yarn run webpack:build`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the jhipster-registry#3.1.0 postinstall script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /Users/xxxxxx/.npm/_logs/2017-08-05T07_44_08_129Z-debug.log
I run:
OS X 10.11.6
node: 8.2.1
npm: 5.3.0
It was a dependency issue. The solution for me was:
Downgrade node.js from version 8 to 6.11 because node-sass didn't support node.js 8. Then I upgraded sass-loader, node-sass and postcss-loader to latest versions.
I don't know if this is the way to go but it work for me.