TargetNotAMemberException while executing a task using distributed executor service - hazelcast - hazelcast

Question

I am trying to use the distributed executor service for hazelcast 3.1 and find that i am unable to use submitToMember(task,member). In my example below 10.69.108.60 is my local machine and 170.194.100.111 is my remote machine. I am able to get return value in my future when the member is my local machine but gives me a TargetNotAMemberException if the member is remote machine.
Below is the code
public class DistExecutionTest {
public static void main(String args[]){
DistributedExecutor dex = new DistributedExecutor();
try {
Member member = new MemberImpl(new Address("170.194.100.111",5701), false );
String msg;
msg = dex.echoOnTheMember("Hey youuuu!", member);
System.out.println(msg);
} catch (UnknownHostException e1) {
e1.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}catch (Exception e) {
e.printStackTrace();
}
}
}
and
public class DistributedExecutor {
Config config;
NetworkConfig network;
JoinConfig join;
DistributedExecutor(){
config = new Config();
network = config.getNetworkConfig();
// network.setPort(5701);
join = network.getJoin();
join.getMulticastConfig().setEnabled(false);
join.getTcpIpConfig().addMember("170.194.100.111").addMember("10.69.108.60").setEnabled(true);
network.getInterfaces().setEnabled(true).addInterface("170.194.100.*").addInterface("10.69.108.*");
}
public String echoOnTheMember(String input, Member member) throws Exception {
Callable<String> task = new DistObject(input);
HazelcastInstance hz = Hazelcast.newHazelcastInstance(config);
IExecutorService executorService = hz.getExecutorService("default");
Future<String> future = executorService.submitToMember(task, member);
String distObjectResult = future.get();
return distObjectResult;
}
}
and
public class Echo implements Callable<String>, Serializable, HazelcastInstanceAware {
private static final long serialVersionUID = -3164053990811643392L;
String message = null;
transient HazelcastInstance localInstance;
public Echo(String msg){
message = msg;
}
#Override
public String call() throws Exception {
return localInstance.toString() + message;
}
#Override
public void setHazelcastInstance(HazelcastInstance hazelcastInstance) {
this.localInstance = hazelcastInstance;
}
}
Here is the logging on local machine
Dec 17, 2013 1:03:20 PM com.hazelcast.instance.DefaultAddressPicker
INFO: Interfaces is enabled, trying to pick one address matching to one of: [162.124.194.*, 10.38.148.*]
Dec 17, 2013 1:03:20 PM com.hazelcast.instance.DefaultAddressPicker
INFO: Prefer IPv4 stack is true.
Dec 17, 2013 1:03:20 PM com.hazelcast.instance.DefaultAddressPicker
INFO: Picked Address[10.69.108.60]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true
Dec 17, 2013 1:03:21 PM com.hazelcast.system
INFO: [10.69.108.60]:5701 [dev] Hazelcast Community Edition 3.1 (20131011) starting at Address[10.69.108.60]:5701
Dec 17, 2013 1:03:21 PM com.hazelcast.system
INFO: [10.69.108.60]:5701 [dev] Copyright (C) 2008-2013 Hazelcast.com
Dec 17, 2013 1:03:21 PM com.hazelcast.instance.Node
INFO: [10.69.108.60]:5701 [dev] Creating TcpIpJoiner
Dec 17, 2013 1:03:21 PM com.hazelcast.core.LifecycleService
INFO: [10.69.108.60]:5701 [dev] Address[10.69.108.60]:5701 is STARTING
Dec 17, 2013 1:03:21 PM com.hazelcast.cluster.TcpIpJoiner
INFO: [10.69.108.60]:5701 [dev] Connecting to possible member: Address[10.69.108.60]:5703
Dec 17, 2013 1:03:21 PM com.hazelcast.cluster.TcpIpJoiner
INFO: [10.69.108.60]:5701 [dev] Connecting to possible member: Address[10.69.108.60]:5702
Dec 17, 2013 1:03:21 PM com.hazelcast.nio.SocketConnector
INFO: [10.69.108.60]:5701 [dev] Connecting to /10.69.108.60:5703, timeout: 0, bind-any: true
Dec 17, 2013 1:03:21 PM com.hazelcast.nio.SocketConnector
INFO: [10.69.108.60]:5701 [dev] Connecting to /10.69.108.60:5702, timeout: 0, bind-any: true
Dec 17, 2013 1:03:21 PM com.hazelcast.cluster.TcpIpJoiner
INFO: [10.69.108.60]:5701 [dev] Connecting to possible member: Address[170.194.100.111]:5703
Dec 17, 2013 1:03:21 PM com.hazelcast.cluster.TcpIpJoiner
INFO: [10.69.108.60]:5701 [dev] Connecting to possible member: Address[170.194.100.111]:5702
Dec 17, 2013 1:03:21 PM com.hazelcast.nio.SocketConnector
INFO: [10.69.108.60]:5701 [dev] Connecting to /170.194.100.111:5703, timeout: 0, bind-any: true
Dec 17, 2013 1:03:21 PM com.hazelcast.nio.SocketConnector
INFO: [10.69.108.60]:5701 [dev] Connecting to /170.194.100.111:5702, timeout: 0, bind-any: true
Dec 17, 2013 1:03:21 PM com.hazelcast.cluster.TcpIpJoiner
INFO: [10.69.108.60]:5701 [dev] Connecting to possible member: Address[170.194.100.111]:5701
Dec 17, 2013 1:03:21 PM com.hazelcast.nio.SocketConnector
INFO: [10.69.108.60]:5701 [dev] Connecting to /170.194.100.111:5701, timeout: 0, bind-any: true
Dec 17, 2013 1:03:22 PM com.hazelcast.nio.SocketConnector
INFO: [10.69.108.60]:5701 [dev] Could not connect to: /10.69.108.60:5703. Reason: SocketException[Connection refused: connect to address /10.69.108.60:5703]
Dec 17, 2013 1:03:22 PM com.hazelcast.nio.SocketConnector
INFO: [10.69.108.60]:5701 [dev] Could not connect to: /10.69.108.60:5702. Reason: SocketException[Connection refused: connect to address /10.69.108.60:5702]
Dec 17, 2013 1:03:22 PM com.hazelcast.nio.SocketConnector
INFO: [10.69.108.60]:5701 [dev] Could not connect to: /170.194.100.111:5703. Reason: SocketException[Connection refused: connect to address /170.194.100.111:5703]
Dec 17, 2013 1:03:22 PM com.hazelcast.nio.SocketConnector
INFO: [10.69.108.60]:5701 [dev] Could not connect to: /170.194.100.111:5702. Reason: SocketException[Connection refused: connect to address /170.194.100.111:5702]
Dec 17, 2013 1:03:22 PM com.hazelcast.nio.SocketConnector
INFO: [10.69.108.60]:5701 [dev] Could not connect to: /170.194.100.111:5701. Reason: SocketException[Connection refused: connect to address /170.194.100.111:5701]
Dec 17, 2013 1:03:23 PM com.hazelcast.cluster.TcpIpJoiner
INFO: [10.69.108.60]:5701 [dev]
Members [1] {
Member [10.69.108.60]:5701 this
}
Dec 17, 2013 1:03:23 PM com.hazelcast.core.LifecycleService
INFO: [10.69.108.60]:5701 [dev] Address[10.69.108.60]:5701 is STARTED
HazelcastInstance{name='_hzInstance_1_dev', node=Address[10.69.108.60]:5701}Hey youuuu!
The logging on the remote machine is on these lines.Couldnt paste all the logging.Managed to get the important part.
INFO: [10.69.108.60]:5701 [dev] Connecting to possible member: Address[10.38.148.60]:5703
Dec 17, 2013 1:03:21 PM com.hazelcast.cluster.TcpIpJoiner
INFO: [10.69.108.60]:5701 [dev] Connecting to possible member: Address[10.38.148.60]:5702
Dec 17, 2013 1:03:21 PM com.hazelcast.cluster.TcpIpJoiner
INFO: [10.69.108.60]:5701 [dev] Connecting to possible member: Address[10.38.148.60]:5701
Dec 17, 2013 1:03:21 PM com.hazelcast.cluster.TcpIpJoiner
INFO: [10.69.108.60]:5701 [dev] Connecting to possible member: Address[170.194.100.111]:5703
Dec 17, 2013 1:03:21 PM com.hazelcast.cluster.TcpIpJoiner
INFO: [10.69.108.60]:5701 [dev] Connecting to possible member: Address[170.194.100.111]:5702
Dec 17, 2013 1:03:21 PM com.hazelcast.nio.SocketConnector
Members [2] {
Member [10.69.108.60]:5701 this
Member [170.194.100.111]:5701
}

Answer 1

Instead of creating a member instance directly, could you get the member instance using the hz.getCluster().getMembers() method and select the one you want to send to? I want to see if it is caused by the way you are creating that member.

Answer 2

I can't format text in comments, so I'll put another answer.
So the problem you are suffering from is why your members don't form a cluster.
You should see logging like:
Members [2] {
Member [192.168.1.104]:5701 this
Member [192.168.1.104]:5702
}
That is why I need more logging that just your stacktrace, but that currently doesn't provide any more value. I need to see what Hazelcast says about joining other clusters.
I need to see something like this:
Dec 17, 2013 7:24:13 PM com.hazelcast.config.XmlConfigBuilder
INFO: Looking for hazelcast.xml config file in classpath.
Dec 17, 2013 7:24:13 PM com.hazelcast.config.XmlConfigBuilder
WARNING: Could not find hazelcast.xml in classpath.
Hazelcast will use hazelcast-default.xml config file in jar.
Dec 17, 2013 7:24:13 PM com.hazelcast.config.XmlConfigBuilder
INFO: Using configuration file /java/projects/Hazelcast/hazelcast/hazelcast/target/classes/hazelcast-default.xml in the classpath.
Dec 17, 2013 7:24:13 PM com.hazelcast.instance.DefaultAddressPicker
INFO: Prefer IPv4 stack is true.
Dec 17, 2013 7:24:13 PM com.hazelcast.instance.DefaultAddressPicker
INFO: Picked Address[192.168.1.102]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true
Dec 17, 2013 7:24:13 PM com.hazelcast.system
INFO: [192.168.1.102]:5701 [dev] [3.2-SNAPSHOT] Hazelcast Community Edition 3.2-SNAPSHOT (20131217) starting at Address[192.168.1.102]:5701
Dec 17, 2013 7:24:13 PM com.hazelcast.system
INFO: [192.168.1.102]:5701 [dev] [3.2-SNAPSHOT] Copyright (C) 2008-2013 Hazelcast.com
Dec 17, 2013 7:24:13 PM com.hazelcast.instance.Node
INFO: [192.168.1.102]:5701 [dev] [3.2-SNAPSHOT] Creating MulticastJoiner
Dec 17, 2013 7:24:13 PM com.hazelcast.core.LifecycleService
INFO: [192.168.1.102]:5701 [dev] [3.2-SNAPSHOT] Address[192.168.1.102]:5701 is STARTING
Dec 17, 2013 7:24:15 PM com.hazelcast.cluster.MulticastJoiner
INFO: [192.168.1.102]:5701 [dev] [3.2-SNAPSHOT]
Members [1] {
Member [192.168.1.102]:5701 this
}
Dec 17, 2013 7:24:16 PM com.hazelcast.core.LifecycleService
INFO: [192.168.1.102]:5701 [dev] [3.2-SNAPSHOT] Address[192.168.1.102]:5701 is STARTED
Dec 17, 2013 7:24:16 PM com.hazelcast.partition.PartitionService
INFO: [192.168.1.102]:5701 [dev] [3.2-SNAPSHOT] Initializing cluster partition table first arrangement...

Answer 3

I was also getting the same error.
It is due to this line in echoonthemember function
HazelcastInstance hz = Hazelcast.newHazelcastInstance(config);
It creates the new hazelcast instance of default or of cfg config. The member is searched in this instance of the hazelcast which is not actually present in it. Thus the error message is displayed as
TargetNotAMemberException.
To have it work properly, just pass the created instance in the echoonthemember function.
E.g by making it the member variable in class DistributedExecutor and setting it by the constructor.
then if your actual instance was 'abcdef', then
use as
IExecutorService executorService = abcdef.getExecutorService("default");
do not create new Hazelcast instance.