I using 4.4.0 datastax-java-driver. In my scenario, I have to provide contacts points as I am connecting to remote cluster. If I am doing so I am getting following error - Since you provided explicit contact points, the local DC must be explicitly set.
I also don't have option to provide this explicitly as I am connecting to different cluster on demand which can be in different data centre. I have found option to set DcInferringLoadBalancingPolicy to infer data centre But I am not sure how to set this in CqlSessionBuilder. Please help me with this.
You need be very careful with it - it's mostly for people who are building tools, such as IDEs, etc. For applications itself it's better to pass datacenter name explicitly - either via config file, or via Java system property.
In short it could be done as following:
ProgrammaticDriverConfigLoaderBuilder configBuilder =
DriverConfigLoader.programmaticBuilder();
configBuilder.withClass(DefaultDriverOption.LOAD_BALANCING_POLICY_CLASS,
DcInferringLoadBalancingPolicy.class);
DriverConfigLoader loader = configBuilder.endProfile().build();
CqlSessionBuilder clusterBuilder = CqlSession.builder()
.addContactPoints(hosts);
CqlSession session = clusterBuilder.withConfigLoader(loader).build();
Related
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 20 days ago.
Improve this question
added config file to take contact-point programatically
#Bean(destroyMethod = "close")
public CqlSession session() {
CqlSession session = CqlSession.builder()
.addContactPoint(InetSocketAddress.createUnresolved("[240b:c0e0:1xx:xxx8:xxxx:x:x:x]", port))
.withConfigLoader(
DriverConfigLoader.programmaticBuilder()
.withString(DefaultDriverOption.LOAD_BALANCING_LOCAL_DATACENTER, localDatacenter) .withString(DefaultDriverOption.AUTH_PROVIDER_USER_NAME,username)
.withString(DefaultDriverOption.AUTH_PROVIDER_PASSWORD,password)
.withString(DefaultDriverOption.CONNECTION_INIT_QUERY_TIMEOUT,"10s")
.withString(DefaultDriverOption.CONNECTION_CONNECT_TIMEOUT, "20s")
.withString(DefaultDriverOption.REQUEST_TIMEOUT, "20s")
.withString(DefaultDriverOption.CONTROL_CONNECTION_TIMEOUT, "20s")
.withString(DefaultDriverOption.SESSION_KEYSPACE,keyspace)
.build())
//.addContactPoint(InetSocketAddress.createUnresolved(InetAddress.getByName(contactPoints).getHostName(), port))
.build();
}
return session;
`Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.datastax.oss.driver.api.core.CqlSession]: Factory method 'cassandraSession' threw exception with message: Since you provided explicit contact points, the local DC must be explicitly set (see basic.load-balancing-policy.local-datacenter in the config, or set it programmatically with SessionBuilder.withLocalDatacenter). Current contact points are: Node(endPoint=/127.0.0.1:9042, hostId=0323221f-9a0f-ec92-ea4a-c1472c2a8b94, hashCode=39075b16)=datacenter1. Current DCs in this cluster are: datacenter1
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:171) ~[spring-beans-6.0.2.jar:6.0.2]
at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:648) ~[spring-beans-6.0.2.jar:6.0.2]
... 89 common frames omitted
Caused by: java.lang.IllegalStateException: Since you provided explicit contact points, the local DC must be explicitly set (see basic.load-balancing-policy.local-datacenter in the config, or set it programmatically with SessionBuilder.withLocalDatacenter). Current contact points are: Node(endPoint=/127.0.0.1:9042, hostId=0323221f-9a0f-ec92-ea4a-c1472c2a8b94, hashCode=39075b16)=datacenter1. Current DCs in this cluster are: datacenter1
at com.datastax.oss.driver.internal.core.loadbalancing.helper.MandatoryLocalDcHelper.discoverLocalDc(MandatoryLocalDcHelper.java:91) ~[java-driver-core-4.11.4-yb-1-RC1.jar:na]
at com.datastax.oss.driver.internal.core.loadbalancing.DefaultLoadBalancingPolicy.discoverLocalDc(DefaultLoadBalancingPolicy.java:119) ~[java-driver-core-4.11.4-yb-1-RC1.jar:na]
at
this is the application.yml file
spring:
data:
cassandra:
keyspace-name: xxx
contact-points: [xxxx:xxxx:xxxx:xxx:xxx:xxx]
port: xxx
local-datacenter: xxxx
use-dc-aware: true
username: xxxxx
password: xxxxx
ssl: true
SchemaAction: CREATE_IF_NOT_EXISTS
but still the application is pointing towards the localhost , even though i've explicitly mentioned the contact-points and localDC
logs of stg evn are :
caused by: com.datastax.oss.driver.api.core.AllNodesFailedException: Could not reach any contact point, make sure you've provided valid addresses (showing first 1 nodes, use getAllErrors() for more): Node (endPoint=/[240b:cOe0:102:xxxx:xxxx:x:x:x]:3xxx,hostId-null,hashCode=4e9ba6a8):[com.datastax.oss.driver.api.core.connection.ConnectionInitException:[s0|controllid:0x984419ed,L:/[240b:cOe0:102:5dd7:
xxxx:x:x:xxx]:4xxx - R:/[240b:c0e0:102:xxxx:xxxx:x:x:x]:3xxx] Protocol initialization request,
step 1 (OPTIONS: unexpected tarlure com.datastax.oss.driver.apt.core.connection.closedconnectiontxception:
Lost connection to remote peer)]
thanks for the question!
I'll try to provide some pointers that might help you identify the problem but it should be noted that you appear to have some non-standard elements to your application. Specifically I note that "java-driver-core-4.11.4-yb-1-RC1.jar" isn't a Java driver artifact released by DataStax (there isn't even a 4.11.4 Java driver release). This could be relevant for reasons we'll get into in a moment. I also don't recognize the configuration file you cite above. Could you provide some more detail on how your app is configured? At first blush it looked as though you might be using spring-data-cassandra but there wasn't any mention of it in your stack trace... so perhaps you're using some kind of custom configuration code?
As to your specific question: my guess is that you might have a Java driver configuration file in your staging environment which is providing a default value for "datastax-java-driver.basic.contact-points". The 4.x Java driver is configured via the Lightbend Config library. Most relevant to our case it searches for a set of configuration files with various default names on the classpath; these files are then merged together to generate the config passed to the driver. So if you have an application.conf in staging which specifies some contact points and is in the classpath the code you cite above would run fine in your local environment but fail in staging.
To validate this, create an application.conf file in your local environment in src/main/resources (or somewhere else that's explicitly included in the classpath) and give it the following contents:
datastax-java-driver {
basic {
contact-points: ["127.0.0.1:9042"]
}
}
If you then re-run the app in your local environment you should see the error there as well.
Note that the core Java driver JAR already includes a reference.conf file which serves as a default configuration. Here's where the part about a custom JAR figures in; because your not using a standard DataStax Java driver JAR I don't know if you're using the standard reference.conf file defined within that JAR. It's possible that the contact points are defined in that file, although if that were the case I'd expect you to already be seeing the error in any environment where you use that JAR.
One final note: the Java driver should be fine with IPv6 addresses. The issue described above isn't related to IPv6; it's entirely a function of how you're using the Java driver's configuration mechanism.
Hopefully some of the above is helpful!
We want to form Hazel cast cluster at server restart. at server start it will find IP address and add member in hazel cast cluster.
please provide any help on approach.
Default configuration is multicast. If this is enabled by your network administrators machines should find each other for you. But it may be
deactivated so you'd need to do another way.
An easy way is TCP discovery,
configuration (if you use YAML) something like:
hazelcast:
network:
join:
multicast:
enabled: false
tcp-ip:
enabled: true
member-list:
- 12.34.56.78:5701
- 34.56.78.12:5701
- 56.78.12.34:5701
When the process starts, it will try those addresses (ignoring if one is itself) to see if anything is there. If it, it'll cluster together.
If you don't know your IPs in advance, you can pass them in as arguments and et them from Java.
JoinConfig joinConfig = config.getNetworkConfig().getJoin();
joinConfig.getMulticastConfig().setEnabled(false);
TcpIpConfig tcpIpConfig = new TcpIpConfig();
tcpIpConfig.setEnabled(true);
tcpIpConfig.setMembers(List.of("12.34.56.78:5701", "34.56.78.12:5701"));
joinConfig.setTcpIpConfig(tcpIpConfig);
We are upgrading datastax Cassandra java driver from 3.2 to 4.x to support DSE 6.8.
Load balancing policies our application currently supports are RoundRobinPolicy and DCAwareRoundRobinPolicy.
These policies aren't available in java-driver-core 4.12.
How can we support the above policies.Please help..
Current code in our application using cassandra-driver-core-3.1.0.jar:
public static LoadBalancingPolicy getLoadBalancingPolicy(String loadBalanceStr, boolean isTokenAware) {
LoadBalancingPolicy loadBalance = null;
if (isTokenAware) {
loadBalance = new TokenAwarePolicy(loadBalanceDataConvert(loadBalanceStr));
} else {
loadBalance = loadBalanceDataConvert(loadBalanceStr);
}
return loadBalance;
}
private static LoadBalancingPolicy loadBalanceDataConvert(String loadBalanceStr) {
if (CassandraConstants.CASSANDRACONNECTION_LOADBALANCEPOLICY_DC.equals(loadBalanceStr)) {
return new DCAwareRoundRobinPolicy.Builder().build();
} else if (CassandraConstants.CASSANDRACONNECTION_LOADBALANCEPOLICY_ROUND.equals(loadBalanceStr)) {
return new RoundRobinPolicy();
}
return null;
}
The load balancing has been heavily simplified in version 4.x of the Cassandra Java driver. You no longer need to nest multiple policies within each other to achieve high availability.
In our opinion, the best policy is the DefaultLoadBalancingPolicy which is enabled by default and achieves all the best attributes as the policies in older versions.
The DefaultLoadBalancingPolicy generates a query plan that is token-aware by default so replicas which own the data appear first and prioritised over other nodes in the local DC. For token-awareness to work, you must provide routing information either by keyspace (with getRoutingKeyspace()), or by routing key (with getRoutingKey()).
If routing information is not provided, the DefaultLoadBalancingPolicy generates a query plan that is a simple round-robin shuffle of available nodes in the local DC.
We understand that developers who are used to configuring DCAwareRoundRobinPolicy in older versions would like to continue using it but we do not recommend it. It is our opinion that failover should take place at the infrastructure layer, not the application layer.
Our opinion is that the DefaultLoadBalancingPolicy is the right choice in all cases. If you prefer to configure DC-failover, make sure you fully understand the implications and know that we think it is the wrong choice.
For details, see the following documents:
Java driver v4 Upgrade Guide
Load Balancing in Java driver v4
I would like to use an Azure Machine Learning Compute Cluster as a compute target but do not want it to containerize my project. Is there a way to deactivate this "feature" ?
The main reasons behind this request is that :
I already set up a docker-compose file that is used to specify 3 containers for Apache Airflow and want to avoid a Docker-in-Docker situation. Especially that I already tried to do so but failed so far (here's the link my other related SO question).
I prefer not to use a Compute Instance as it is tied to an Azure account which is not ideal for automation purposes.
Thanks in advance !
Use the provisioning_configuration method of the AmlCompute class to specify configuration parameters.
In the following example, a persistent compute target provisioned by AmlCompute is created. The provisioning_configuration parameter in this example is of type AmlComputeProvisioningConfiguration, which is a child class of ComputeTargetProvisioningConfiguration.
from azureml.core.compute import ComputeTarget, AmlCompute
from azureml.core.compute_target import ComputeTargetException
# Choose a name for your CPU cluster
cpu_cluster_name = "cpu-cluster"
# Verify that cluster does not exist already
try:
cpu_cluster = ComputeTarget(workspace=ws, name=cpu_cluster_name)
print('Found existing cluster, use it.')
except ComputeTargetException:
compute_config = AmlCompute.provisioning_configuration(vm_size='STANDARD_D2_V2',
max_nodes=4)
cpu_cluster = ComputeTarget.create(ws, cpu_cluster_name, compute_config)
cpu_cluster.wait_for_completion(show_output=True)
Refer - https://learn.microsoft.com/en-us/python/api/azureml-core/azureml.core.compute.amlcompute(class)?view=azure-ml-py
I'm currently playing with Hazelcast Cloud. My use case requires me to upload 50mb of jar file dependencies to Hazelcast Cloud servers. I found out that the upload seems to give up after about a minute or so. I get an upload rate of about 1mb a second, it drops after a while and then stops. I have repeated it a few times and the same thing happens.
Here is the config code I'm using:
Clientconfig config = new ClientConfig();
ClientUserCodeDeploymentConfig clientUserCodeDeploymentConfig =
new ClientUserCodeDeploymentConfig();
// added many jars here...
clientUserCodeDeploymentConfig.addJar("jar dependancy path..");
clientUserCodeDeploymentConfig.addJar("jar dependancy path..");
clientUserCodeDeploymentConfig.addJar("jar dependancy path..");
clientUserCodeDeploymentConfig.setEnabled(true);
config.setUserCodeDeploymentConfig(clientUserCodeDeploymentConfig);
ClientNetworkConfig networkConfig = new ClientNetworkConfig();
networkConfig.setConnectionTimeout(9999999); // i.e. don't timeout
networkConfig.setConnectionAttemptPeriod(9999999); // i.e. don't timeout
config.setNetworkConfig(networkConfig);
Any idea what's the cause, maybe there's a limit on the free cloud cluster?
I'd suggest using the smaller jar because this feature, the client user code upload, was designed for a bit different use cases:
You have objects that run on the cluster via the clients such as Runnable, Callable and Entry Processors.
You have new or amended user domain objects (in-memory format of the IMap set to Object) which need to be deployed into the cluster.
Please see more info here.