Why isn't my Eureka server being found by Hazelcast-Eureka-One plugin? - hazelcast

Question

I have a working Hazelcast cluster configured with tcp-ip. I need it to work with Eureka discovery. I am trying to implement the hazelcast-eureka-one plugin.
The (Spring-Boot) app currently already registers itself with Eureka sucessfully, using the #EnableEurekaClient annotation. I am not concerned with whether the hazelcast eureka client is the same or a different client. I am fine with hazelcast registering itself separately from the app. As long as it works.
When I remove eureka-client.properties, the app will not start up, showing an error that eureka-client.properties can not be found. When I have the file in place, the app starts, but apparently none of the properties from eureka-client.properties are being loaded, which leaves hazelcast not knowing where the eureka server is. The logs indicate that the properties file is being found, but none of the properties seem to be imported.
Upgrading hazelcast-eureka-one to 1.1 makes no change.
Setting use-metadata-for-host-and-port to true makes no change.
Gradle:
compile group: 'com.hazelcast', name: 'hazelcast-spring', version: '3.9.4'
compile group: 'com.hazelcast', name: 'hazelcast-hibernate52', version: '1.2.3'
compile group: 'com.hazelcast', name: 'hazelcast-eureka-one', version: '1.0.1'
hazelcast.xml:
<?xml version="1.0" encoding="UTF-8"?>
<hazelcast xsi:schemaLocation="http://www.hazelcast.com/schema/config http://www.hazelcast.com/schema/config/hazelcast-config-3.9.xsd"
xmlns="http://www.hazelcast.com/schema/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<instance-name>app.name.hazelcast.sessions-instance</instance-name>
<group>
<name>app.name.hazelcast.sessions.local-group</name>
</group>
<network>
<join>
<multicast enabled="false"/>
<tcp-ip enabled="false"/>
<aws enabled="false"/>
<discovery-strategies>
<discovery-strategy class="com.hazelcast.eureka.one.EurekaOneDiscoveryStrategy" enabled="true">
<properties>
<property name="self-registration">true</property>
<property name="namespace">hazelcast-app-name</property>
<property name="use-metadata-for-host-and-port">false</property>
</properties>
</discovery-strategy>
</discovery-strategies>
</join>
</network>
<map name="spring:session:sessions">
<attributes>
<attribute extractor="org.springframework.session.hazelcast.PrincipalNameExtractor">principalName</attribute>
</attributes>
<indexes>
<index>principalName</index>
</indexes>
</map>
eureka-client.properties:
hazelcast.shouldUseDns=false
hazelcast.datacenter=primary
hazelcast.name=hazelcast-app-name-sessions
hazelcast.serviceUrl.default=http://username:password#svcregistry1-dev.company.com:8580/eureka/,http://username:password#svcregistry2-dev.company.com:8590/eureka/
Log file:
Loading 'hazelcast.xml' from classpath.
2019-02-15 11:19:13,935 - INFO - [localhost-startStop-1] - [,,] - com.hazelcast.instance.AddressPicker : [LOCAL] [app.name.hazelcast.sessions.local-group] [3.9.4] Prefer IPv4 stack is true.
2019-02-15 11:19:14,166 - INFO - [localhost-startStop-1] - [,,] - com.hazelcast.instance.AddressPicker : [LOCAL] [app.name.hazelcast.sessions.local-group] [3.9.4] Picked [172.28.208.1]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true
2019-02-15 11:19:14,179 - INFO - [localhost-startStop-1] - [,,] - com.hazelcast.system : [172.28.208.1]:5701 [app.name.hazelcast.sessions.local-group] [3.9.4] Hazelcast 3.9.4 (20180420 - b8001d5) starting at [172.28.208.1]:5701
2019-02-15 11:19:14,179 - INFO - [localhost-startStop-1] - [,,] - com.hazelcast.system : [172.28.208.1]:5701 [app.name.hazelcast.sessions.local-group] [3.9.4] Copyright (c) 2008-2018, Hazelcast, Inc. All Rights Reserved.
2019-02-15 11:19:14,179 - INFO - [localhost-startStop-1] - [,,] - com.hazelcast.system : [172.28.208.1]:5701 [app.name.hazelcast.sessions.local-group] [3.9.4] Configured Hazelcast Serialization version: 1
2019-02-15 11:19:14,616 - INFO - [localhost-startStop-1] - [,,] - c.h.s.i.o.impl.BackpressureRegulator : [172.28.208.1]:5701 [app.name.hazelcast.sessions.local-group] [3.9.4] Backpressure is disabled
2019-02-15 11:19:15,309 - DEBUG - [localhost-startStop-1] - [,,] - .n.c.u.OverridingPropertiesConfiguration : Base path set to file:///C:/Users/my.name/IdeaProjects/AppName/build/classes/main/
2019-02-15 11:19:15,310 - DEBUG - [localhost-startStop-1] - [,,] - .n.c.u.OverridingPropertiesConfiguration : FileName set to eureka-client.properties
2019-02-15 11:19:15,310 - DEBUG - [localhost-startStop-1] - [,,] - .n.c.u.OverridingPropertiesConfiguration : URL set to file:/C:/Users/my.name/IdeaProjects/AppName/build/classes/main/eureka-client.properties
2019-02-15 11:19:15,316 - INFO - [localhost-startStop-1] - [,,] - c.n.config.util.ConfigurationUtils : Loaded properties file file:/C:/Users/my.name/IdeaProjects/AppName/build/classes/main/eureka-client.properties
2019-02-15 11:19:15,326 - INFO - [localhost-startStop-1] - [,,] - .p.EurekaConfigBasedInstanceInfoProvider : Setting initial instance status as: STARTING
2019-02-15 11:19:15,334 - WARN - [localhost-startStop-1] - [,,] - c.n.config.util.ConfigurationUtils : file:/C:/Users/my.name/IdeaProjects/AppName/build/classes/main/eureka-client.properties is already loaded
2019-02-15 11:19:15,385 - INFO - [localhost-startStop-1] - [,,] - com.netflix.discovery.DiscoveryClient : Initializing Eureka in region us-east-1
2019-02-15 11:19:15,951 - INFO - [localhost-startStop-1] - [,,] - c.n.d.provider.DiscoveryJerseyProvider : Using JSON encoding codec LegacyJacksonJson
2019-02-15 11:19:15,951 - INFO - [localhost-startStop-1] - [,,] - c.n.d.provider.DiscoveryJerseyProvider : Using JSON decoding codec LegacyJacksonJson
2019-02-15 11:19:16,143 - INFO - [localhost-startStop-1] - [,,] - c.n.d.provider.DiscoveryJerseyProvider : Using XML encoding codec XStreamXml
2019-02-15 11:19:16,143 - INFO - [localhost-startStop-1] - [,,] - c.n.d.provider.DiscoveryJerseyProvider : Using XML decoding codec XStreamXml
2019-02-15 11:19:16,392 - INFO - [localhost-startStop-1] - [,,] - c.n.d.s.r.aws.ConfigClusterResolver : Resolving eureka endpoints via configuration
2019-02-15 11:19:16,394 - DEBUG - [localhost-startStop-1] - [,,] - c.n.discovery.endpoint.EndpointUtils : The availability zone for the given region us-east-1 are [defaultZone]
2019-02-15 11:19:16,394 - DEBUG - [localhost-startStop-1] - [,,] - c.n.d.s.r.aws.ConfigClusterResolver : Config resolved to []
2019-02-15 11:19:16,394 - ERROR - [localhost-startStop-1] - [,,] - c.n.d.s.r.aws.ConfigClusterResolver : Cannot resolve to any endpoints from provided configuration: {defaultZone=[]}
2019-02-15 11:19:16,612 - DEBUG - [localhost-startStop-1] - [,,] - c.n.d.s.r.a.ZoneAffinityClusterResolver : Local zone=defaultZone; resolved to: []
2019-02-15 11:19:16,612 - ERROR - [localhost-startStop-1] - [,,] - c.n.d.s.transport.EurekaHttpClients : Initial resolution of Eureka server endpoints failed. Check ConfigClusterResolver logs for more info
2019-02-15 11:19:16,647 - INFO - [localhost-startStop-1] - [,,] - com.netflix.discovery.DiscoveryClient : Disable delta property : false
2019-02-15 11:19:16,647 - INFO - [localhost-startStop-1] - [,,] - com.netflix.discovery.DiscoveryClient : Single vip registry refresh property : null
2019-02-15 11:19:16,647 - INFO - [localhost-startStop-1] - [,,] - com.netflix.discovery.DiscoveryClient : Force full registry fetch : false
2019-02-15 11:19:16,647 - INFO - [localhost-startStop-1] - [,,] - com.netflix.discovery.DiscoveryClient : Application is null : false
2019-02-15 11:19:16,647 - INFO - [localhost-startStop-1] - [,,] - com.netflix.discovery.DiscoveryClient : Registered Applications size is zero : true
2019-02-15 11:19:16,647 - INFO - [localhost-startStop-1] - [,,] - com.netflix.discovery.DiscoveryClient : Application version is -1: true
2019-02-15 11:19:16,647 - INFO - [localhost-startStop-1] - [,,] - com.netflix.discovery.DiscoveryClient : Getting all instance registry info from the eureka server
2019-02-15 11:19:16,648 - DEBUG - [localhost-startStop-1] - [,,] - c.n.d.s.t.d.SessionedEurekaHttpClient : Ending a session and starting anew
2019-02-15 11:19:16,655 - ERROR - [localhost-startStop-1] - [,,] - com.netflix.discovery.DiscoveryClient : DiscoveryClient_UNKNOWN/0c99d08b-8072-4fe4-a20f-c8653e10e374 - was unable to refresh its cache! status = There is no known eureka server; cluster server list is empty
com.netflix.discovery.shared.transport.TransportException: There is no known eureka server; cluster server list is empty
at com.netflix.discovery.shared.transport.decorator.RetryableEurekaHttpClient.execute(RetryableEurekaHttpClient.java:108)
at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.getApplications(EurekaHttpClientDecorator.java:134)
at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$6.execute(EurekaHttpClientDecorator.java:137)
at com.netflix.discovery.shared.transport.decorator.SessionedEurekaHttpClient.execute(SessionedEurekaHttpClient.java:77)
at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.getApplications(EurekaHttpClientDecorator.java:134)
at com.netflix.discovery.DiscoveryClient.getAndStoreFullRegistry(DiscoveryClient.java:1051)
at com.netflix.discovery.DiscoveryClient.fetchRegistry(DiscoveryClient.java:965)
at com.netflix.discovery.DiscoveryClient.<init>(DiscoveryClient.java:414)
at com.netflix.discovery.DiscoveryClient.<init>(DiscoveryClient.java:269)
at com.netflix.discovery.DiscoveryClient.<init>(DiscoveryClient.java:265)
at com.netflix.discovery.DiscoveryClient.<init>(DiscoveryClient.java:257)
at com.hazelcast.eureka.one.EurekaOneDiscoveryStrategy.<init>(EurekaOneDiscoveryStrategy.java:147)
at com.hazelcast.eureka.one.EurekaOneDiscoveryStrategy.<init>(EurekaOneDiscoveryStrategy.java:55)
at com.hazelcast.eureka.one.EurekaOneDiscoveryStrategy$EurekaOneDiscoveryStrategyBuilder.build(EurekaOneDiscoveryStrategy.java:111)
at com.hazelcast.eureka.one.EurekaOneDiscoveryStrategyFactory.newDiscoveryStrategy(EurekaOneDiscoveryStrategyFactory.java:53)
at com.hazelcast.spi.discovery.impl.DefaultDiscoveryService.buildDiscoveryStrategy(DefaultDiscoveryService.java:185)
at com.hazelcast.spi.discovery.impl.DefaultDiscoveryService.loadDiscoveryStrategies(DefaultDiscoveryService.java:145)
at com.hazelcast.spi.discovery.impl.DefaultDiscoveryService.<init>(DefaultDiscoveryService.java:60)
at com.hazelcast.spi.discovery.impl.DefaultDiscoveryServiceProvider.newDiscoveryService(DefaultDiscoveryServiceProvider.java:29)
at com.hazelcast.instance.Node.createDiscoveryService(Node.java:265)
at com.hazelcast.instance.Node.<init>(Node.java:216)
at com.hazelcast.instance.HazelcastInstanceImpl.createNode(HazelcastInstanceImpl.java:160)
at com.hazelcast.instance.HazelcastInstanceImpl.<init>(HazelcastInstanceImpl.java:128)
at com.hazelcast.instance.HazelcastInstanceFactory.constructHazelcastInstance(HazelcastInstanceFactory.java:195)
at com.hazelcast.instance.HazelcastInstanceFactory.newHazelcastInstance(HazelcastInstanceFactory.java:174)
at com.hazelcast.instance.HazelcastInstanceFactory.newHazelcastInstance(HazelcastInstanceFactory.java:124)
at com.hazelcast.core.Hazelcast.newHazelcastInstance(Hazelcast.java:92)
at org.springframework.boot.autoconfigure.hazelcast.HazelcastServerConfiguration$HazelcastServerConfigFileConfiguration.hazelcastInstance(HazelcastServerConfiguration.java:56)
at org.springframework.boot.autoconfigure.hazelcast.HazelcastServerConfiguration$HazelcastServerConfigFileConfiguration$$EnhancerBySpringCGLIB$$d6cfebe6.CGLIB$hazelcastInstance$0(<generated>)
at org.springframework.boot.autoconfigure.hazelcast.HazelcastServerConfiguration$HazelcastServerConfigFileConfiguration$$EnhancerBySpringCGLIB$$d6cfebe6$$FastClassBySpringCGLIB$$3a3e2869.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:365)
at org.springframework.boot.autoconfigure.hazelcast.HazelcastServerConfiguration$HazelcastServerConfigFileConfiguration$$EnhancerBySpringCGLIB$$d6cfebe6.hazelcastInstance(<generated>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154)
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:583)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1246)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1096)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:535)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:495)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:317)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:315)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:251)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1135)
at org.springframework.beans.factory.support.DefaultListableBeanFactory$DependencyObjectProvider.getObject(DefaultListableBeanFactory.java:1665)
at org.springframework.session.hazelcast.config.annotation.web.http.HazelcastHttpSessionConfiguration.setHazelcastInstance(HazelcastHttpSessionConfiguration.java:96)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredMethodElement.inject(AutowiredAnnotationBeanPostProcessor.java:696)
at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:90)
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:370)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1336)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:572)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:495)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:317)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:315)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:373)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1246)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1096)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:535)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:495)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:317)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:315)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:251)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1135)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1062)
at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:819)
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:725)
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:475)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1246)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1096)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:535)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:495)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:317)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:315)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:204)
at org.springframework.boot.web.servlet.ServletContextInitializerBeans.getOrderedBeansOfType(ServletContextInitializerBeans.java:226)
at org.springframework.boot.web.servlet.ServletContextInitializerBeans.getOrderedBeansOfType(ServletContextInitializerBeans.java:214)
at org.springframework.boot.web.servlet.ServletContextInitializerBeans.addServletContextInitializerBeans(ServletContextInitializerBeans.java:91)
at org.springframework.boot.web.servlet.ServletContextInitializerBeans.<init>(ServletContextInitializerBeans.java:80)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.getServletContextInitializerBeans(ServletWebServerApplicationContext.java:250)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.selfInitialize(ServletWebServerApplicationContext.java:237)
at org.springframework.boot.web.embedded.tomcat.TomcatStarter.onStartup(TomcatStarter.java:54)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5245)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1420)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1410)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Answer 1

Your namespace in hazelcast.xml must be the same as the prefix of the properties in eureka-client.properties.
In other words, you need to either change the namespace to:
<property name="namespace">hazelcast</property>
Or change your eureka-client.properties to:
hazelcast-app-name.shouldUseDns=false
hazelcast-app-name.datacenter=primary
hazelcast-app-name.name=hazelcast-app-name-sessions
hazelcast-app-name.serviceUrl.default=http://username:password#svcregistry1-dev.company.com:8580/eureka/,http://username:password#svcregistry2-dev.company.com:8590/eureka/
Please read more at:
Hazelcast Eureka Plugin GH repository
Hazelcast Eureka Plugin Code Sample