I am trying to build a 3 node Cassandra cluster on Centos 6.7. I had no issues installing it and I can successfully run the cqlshell commands. However whenever I try to run the stress test as very simple testing case as:
cassandra-stress write n=10
I get this error
java.lang.RuntimeException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.TransportException: [localhost/127.0.0.1] Cannot connect))
at org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:222)
at org.apache.cassandra.stress.settings.SettingsSchema.createKeySpacesNative(SettingsSchema.java:79)
at org.apache.cassandra.stress.settings.SettingsSchema.createKeySpaces(SettingsSchema.java:69)
at org.apache.cassandra.stress.settings.StressSettings.maybeCreateKeyspaces(StressSettings.java:230)
at org.apache.cassandra.stress.StressAction.run(StressAction.java:54)
at org.apache.cassandra.stress.Stress.run(Stress.java:133)
at org.apache.cassandra.stress.Stress.main(Stress.java:61)
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.TransportException: [localhost/127.0.0.1] Cannot connect))
at com.datastax.driver.core.ControlConnection.reconnectInternal(ControlConnection.java:231)
at com.datastax.driver.core.ControlConnection.connect(ControlConnection.java:77)
at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1414)
at com.datastax.driver.core.Cluster.getMetadata(Cluster.java:393)
at org.apache.cassandra.stress.util.JavaDriverClient.connect(JavaDriverClient.java:154)
at org.apache.cassandra.stress.settings.StressSettings.getJavaDriverClient(StressSettings.java:213)
... 6 more
Both the seeds, listen_address and rpc_address have the correct addresses and I opened 9042, 7000, 7001 and 7199 ports in the firewall. So I am not entirely sure what might be the problem.
run nodetool status
Take one of the printed IP addresses and try: cassandra-stress write n=10 -node <ip address>
Related
I'm currently trying to connect to a Apache Cassandra database using Apache Spark (2.3.0,shell) using the Datastax driver (datastax:spark-cassandra-connector:2.3.0-s_2.11).
I'm using the --conf option at the command line and when I try to run a database query its erroring out saying that it cant open a native connection to 127.0.0.1:9042.
Step 1 (I'm running this command inside the folder where spark is.)
# ./bin/spark-shell --conf spark.cassandra-connection.host=localhost spark.cassandra-connection.native.port=32771 --packages datastax:spark-cassandra-connector:2.3.0-s_2.11
Step 2 (Im Running these steps in the scala> shell of Spark)
scala> import com.datastax.spark.connector._
scala> import org.apache.spark.sql.cassandra._
scala> val rdd = sc.cassandraTable("market", "markethistory")
scala> println(rdd.first)
Step 3 (It Errors out)
java.io.IOException: Failed to open native connection to Cassandra at {127.0.0.1}:9042 +stacktrace
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.TransportException: [localhost/127.0.0.1:9042] Cannot connect)) +stacktrace
Additional notes:
Notice how it says port 9042 in the error.
I've also tried changing the host in the --conf option and that doesn't change the output of the error.
My main assumption would be that I need to specify the host and port in scala but I'm unsure how, and the datastax documentation is all about their special spark distro and it doesn't seem to match up.
Things I've tried:
spark.cassandra-connection.port=32771
spark.cassandra.connection.port=32771
spark.cassandra.connection.host=localhost
Thanks in advance.
The Answer was twofold;
The strings are indeed cassandra.connection not cassandra-connection
--conf has to be after --packages
Thanks to #user8371915 for the connection string difference.
I'm able to connect to cassandra docker instance using external ip with CQLSH but it is failing to connect from java client.
Server logs:
Server.java:156 - Starting listening for CQL clients on /0.0.0.0:9042 (unencrypted)...
Error from application:
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /14.24.34.44:9042 (com.datastax.driver.core.exceptions.TransportException: [/14.24.34.44:9042] Cannot connect))
Cassandra version used: 3.11.1
I have installed Cassandra v2.0.16 on CentOS release 6.6 (Final).
But when I try to do cqlsh on it, It gives the following error,
Connection error: Request did not complete within rpc_timeout
On My local machine, I am running ubuntu 12.04, it works fine, but on my production machine (User Managed VPS running Cent OS) its not working properly.
I went to the /var/log/cassandra/system.log
ERROR [main] 2015-07-02 04:42:12,973 CassandraDaemon.java (line 571) Exception encountered during startup
org.jboss.netty.channel.ChannelException: Failed to bind to: localhost/127.0.0.1:9042
at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272)
at org.apache.cassandra.transport.Server.run(Server.java:159)
at org.apache.cassandra.transport.Server.start(Server.java:110)
at org.apache.cassandra.service.CassandraDaemon.start(CassandraDaemon.java:489)
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:567)
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:643)
Caused by: java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:444)
at sun.nio.ch.Net.bind(Net.java:436)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.jboss.netty.channel.socket.nio.NioServerBoss$RegisterTask.run(NioServerBoss.java:193)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:366)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:290)
at org.jboss.netty.channel.socket.nio.NioServerBoss.run(NioServerBoss.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
I am just stuck here.
Please any help would be appreciated.
I am not getting how to solve this problem.
Note: I have reinstalled Cassandra 2 to 3 times. But stuck with same problem
When I'm trying to connect to the cassandra seed node using the datastax connector, I can't.
I have four spark nodes: one master and three workers. This works well on its own. The same machines have cassandra installed on them with the one being the spark master as a seed node. This works on its own as well (I successfully wrote and read from it).
Now, I'm trying to do
val info = spark_context.cassandraTable("files", "metainfo")
println( info.count )
Before, I specify the spark context as follows:
val confStandalone = new SparkConf()
.set("spark.cassandra.connection.host", "10.14.56.156")
.setMaster("spark://10.14.56.156:7077")
.setAppName("Test")
.set("spark.executor.memory", "1g")
.set("spark.eventLog.enabled", "true")
.set("spark.driver.host", "10.14.56.156")
.set("spark.broadcast.factory", "org.apache.spark.broadcast.HttpBroadcastFactory")
val spark_context = new SparkContext( confStandalone )
spark_context.addJar("SOME_PATH/spark-cassandra-connector_2.10-1.2.0-alpha1.jar")
In the cassandra.yaml file I set the rpc_address to 10.14.56.156 and used the standard ports (9160, 9042). Now when I do
sbt run
I get the following error:
15/03/18 16:38:43 INFO LocalNodeFirstLoadBalancingPolicy: Adding host 127.0.0.1 (datacenter1)
15/03/18 16:38:43 INFO LocalNodeFirstLoadBalancingPolicy: Adding host 10.14.56.156 (datacenter1)
15/03/18 16:38:43 INFO LocalNodeFirstLoadBalancingPolicy: Adding host 127.0.0.1 (datacenter1)
15/03/18 16:38:43 ERROR Session: Error creating pool to /127.0.0.1:9042 com.datastax.driver.core.TransportException: [/127.0.0.1:9042] Cannot connect
at com.datastax.driver.core.Connection.<init>(Connection.java:106)
at com.datastax.driver.core.PooledConnection.<init>(PooledConnection.java:35)
at com.datastax.driver.core.Connection$Factory.open(Connection.java:528)
...
Caused by: java.net.ConnectException: Connection refused: /127.0.0.1:9042 at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
...
Now, when I change the rpc_address to 0.0.0.0 as id sometimes advised, I get the same error but with 10.14.56.156 instead of 127.0.0.1 and only the line:
15/03/18 16:38:43 INFO LocalNodeFirstLoadBalancingPolicy: Adding host 10.14.56.156 (datacenter1)
with the one above and the one below (referring to 127.0.0.1) removed.
I didn't set any firewall rules in the iptables, so I don't think that would be an issue. Help appreciated!
Have you looked at what broadcast_rpc_address is set to? The java-driver will derive the ip to connect to from the 'peer' column of system.peers. If rpc_address is set to 0.0.0.0, broadcast_rpc_address must be set.
My guess is that with your rpc_address set to 0.0.0.0, the driver is connecting from the broadcast_rpc_address even though it says [/10.14.56.156:9042] Cannot connect (you may see Connection refused: /127.0.0.1:9042 further in the stack trace).
When i am using Cassandra's nodetool to see the ring of a remote host (use IP address), it gives following error, how to make this work?
BTW - I can ping that host using IP address.
root#ServerA:~/cassandra# bin/nodetool -h 172.24.0.10 ring
Error connecting to remote JMX agent!
java.rmi.ConnectException: Connection refused to host: 127.0.1.1; nested exception is:
java.net.ConnectException: Connection refused
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:128)
at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2343)
at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:296)
at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:267)
at org.apache.cassandra.tools.NodeProbe.connect(NodeProbe.java:106)
at org.apache.cassandra.tools.NodeProbe.(NodeProbe.java:82)
at org.apache.cassandra.tools.NodeCmd.main(NodeCmd.java:405)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:310)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:176)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:163)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:384)
at java.net.Socket.connect(Socket.java:546)
at java.net.Socket.connect(Socket.java:495)
at java.net.Socket.(Socket.java:392)
at java.net.Socket.(Socket.java:206)
at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:40)
at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:146)
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:613)
... 10 more
You need to enable remote JMX:
To enable monitoring and management
from remote systems, set this system
property when you start the JVM:
com.sun.management.jmxremote.port=portNum