My Error is as below
OpsCenter was not able to add the cluster: OpsCenter was unable to resolve the ip for Test_Cluster, please specify seed nodes using the rpc_address
My OS is CentOS 7
I install DSE 6
i found that datastax forbidden my ip
Related
I created a spark cluster based in this link.
Everything went smooth but the problem is after the cluster created im trying to use pyspark to connect remotely to the container inside the host from other machine.
I'm receiving a 18/04/04 17:14:48 WARN StandaloneAppClient$ClientEndpoint: Failed to connect to master xxxx.xxxx:7077 even though i can connect through telnet to the 7077 port from that host!
What may i be missing out?
I have a dse cluster base on docker. Now I want to use Mesos to manage my dse cluster for running spark job. So I installed Mesos master, Mesos slave docker container. What should I do next to connect dse to mesos?
I am trying to bring up datastax cassandra in analytics mode by using "dse cassandra -k -s". I am using DSE 5.0 sandbox on a single node setup.
I have configured the spark-env.sh with SPARK_MASTER_IP as well as SPARK_LOCAL_IP to point to my LAN IP.
export SPARK_LOCAL_IP="172.40.9.79"
export SPARK_MASTER_HOST="172.40.9.79"
export SPARK_WORKER_HOST="172.40.9.79"
export SPARK_MASTER_IP="172.40.9.79"
All above variables are setup in spark-env.sh.
Despite these, the worker will not come up. It is always looking for a master at 127.0.0.1.This is the error i am seeing in /var/log/cassandra/system.log
WARN [worker-register-master-threadpool-8] 2016-10-04 08:02:45,832 SPARK-WORKER Logging.scala:91 - Failed to connect to master 127.0.0.1:7077
java.io.IOException: Failed to connect to /127.0.0.1:7077
Result from dse client-tool shows 127.0.0.1
$ dse client-tool -u cassandra -p cassandra spark master-address
spark://127.0.0.1:7077
However i am able to access the spark web UI from the LAN IP 172.40.9.79
Spark Web UI screenshot
Any help is greatly appreciated
Try add in file spark-defaults.conf this parameter: spark.master local[*] or spark.master 172.40.9.79. Maybe this solves your problem
All,
I have a 3 noded cluster cassandra in Digital Ocean . the version of cassandra as per SHOW VERSION in CQL is shown below
[cqlsh 5.0.1 | Cassandra 3.0.0 | CQL spec 3.3.1 | Native protocol v4]
I am able to connnect to one node of the cluster from another node using cqlsh and run commands... However when i try to connect using the java driver , i get the following exception
com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /128.xxx.xxx.xx:9042 (com.datastax.driver.core.TransportException: [/128.xxx.xxx.xxx:9042] Connection has been closed))
at com.datastax.driver.core.ControlConnection.reconnectInternal(ControlConnection.java:222)
at com.datastax.driver.core.ControlConnection.connect(ControlConnection.java:77)
at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1232)
at com.datastax.driver.core.Cluster.getMetadata(Cluster.java:336)
at com.attinad.cantiz.iot.platform.vehicledatapoc.App.connect(App.java:22)
at com.attinad.cantiz.iot.platform.vehicledatapoc.App.main(App.java:14)
The version of java driver that i am using is 2.0.10. The maven configuration is given below
<dependency>
<groupId>com.datastax.cassandra</groupId>
<artifactId>cassandra-driver-core</artifactId>
<version>2.0.10</version>
</dependency>
I checked the cassandra.yaml and found that the following settings has been set correctly
start_native_transport: true
native_transport_port: 9042
rpc_address: 128.xxx.xxx.xx
listen_address: 128.xxx.xxx.xx
has been configured correctly... So i am completely lost... Any help is appreciated
According to the 2.0.10 driver documentation, that version of the driver is compatible with Apache Cassandra 1.2 and 2.0. Compatibility with 3.0 is added in the 3.0 driver, which is currently at 3.0.0-beta1. The protocol compatibility error should be shown in the Cassandra server logs.
You could either downgrade Cassandra to a 2.x version or try out the beta driver. Downgrading Cassandra should be the safer choice if you want to use the system in production now.
Ran into this myself... had tried Cassandra 3.0.0 with driver 2.1.9.
Fixed it by going to driver 3.0.0.
I have a DSE 4.5 installation with spark running. I need some help in passing the username / password of cassandra cluster from Spark Shell.
I have added these properties to conf/spark-default.conf file
spark.cassandra.auth.username=user
spark.cassandra..auth.password=pass
And start up my spark shell using
dse spark
But still seeing the error when I try sc.cassandraTable
com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host /11.111.11.11:9042: Host /11.111.11.11:9042 requires authentication, but no authenticator found in Cluster configuration
at com.datastax.driver.core.AuthProvider$1.newAuthenticator(AuthProvider.java:38)
at com.datastax.driver.core.Connection.initializeTransport(Connection.java:138)
at com.datastax.driver.core.Connection.<init>(Connection.java:111)
at com.datastax.driver.core.Connection$Factory.open(Connection.java:432)
at com.datastax.driver.core.ControlConnection.tryConnect(ControlConnection.java:216)
at com.datastax.driver.core.ControlConnection.reconnectInternal(ControlConnection.java:171)
at com.datastax.driver.core.ControlConnection.connect(ControlConnection.java:79)
at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1104)
looks like you can execute this command
dse spark -Dcassandra.username=user -Dcassandra.password=pass
ref:
http://docs.datastax.com/en/datastax_enterprise/4.5/datastax_enterprise/sec/secIntrnlAuth.html?scroll=secItrnlAuth__authentication-for-hadoop-tools
This worked for me:
dse -u cassandra -p cassandra spark