cassandra 1.2 error in startup with ssl enabled - cassandra

I am using cassandra 1.2 datastax community edition. I am trying to enable the client_encryption_options and the server is not starting up. I am getting the below exception.
cassnadra.yaml configuration
client_encryption_options:
enabled: true
keystore: C:/Projects/MainApp/cassandrakeystore.jks
keystore_password: cassandra
require_client_auth: true
# Set trustore and truststore_password if require_client_auth is true
truststore: C:/Projects/MainApp/cassandratruststore.jks
truststore_password: cassandra
Exception on the startup
ERROR [main] 2014-07-08 10:41:09,006 CassandraDaemon.java (line 464) Exception encountered during startup
java.lang.RuntimeException: Unable to create thrift socket to localhost/127.0.0.1:9160
at org.apache.cassandra.thrift.CustomTThreadPoolServer$Factory.buildTServer(CustomTThreadPoolServer.java:267)
at org.apache.cassandra.thrift.TServerCustomFactory.buildTServer(TServerCustomFactory.java:46)
at org.apache.cassandra.thrift.ThriftServer$ThriftServerThread.<init>(ThriftServer.java:105)
at org.apache.cassandra.thrift.ThriftServer.start(ThriftServer.java:52)
at org.apache.cassandra.service.CassandraDaemon.start(CassandraDaemon.java:400)
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:460)
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:490)
Caused by: org.apache.thrift.transport.TTransportException: Could not bind to port 9160
at org.apache.thrift.transport.TSSLTransportFactory.createServer(TSSLTransportFactory.java:117)
at org.apache.thrift.transport.TSSLTransportFactory.getServerSocket(TSSLTransportFactory.java:103)
at org.apache.cassandra.thrift.CustomTThreadPoolServer$Factory.buildTServer(CustomTThreadPoolServer.java:257)
... 6 more
Caused by: java.lang.IllegalArgumentException: Cannot support TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA with currently installed providers
at com.sun.net.ssl.internal.ssl.CipherSuiteList.<init>(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLServerSocketImpl.setEnabledCipherSuites(Unknown Source)
at org.apache.thrift.transport.TSSLTransportFactory.createServer(TSSLTransportFactory.java:113)
... 8 more

At a glance, the following line makes me think you need to install JCE:
Caused by: java.lang.IllegalArgumentException: Cannot support TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA with currently installed providers
See: http://www.oracle.com/technetwork/java/javase/downloads/jce-7-download-432124.html

Related

Apache Zeppelin - Zeppelin tutorial failed to create interpreter - Connection refused

I am trying to test Zeppelin 0.6.2 with a Spark 2.0.1 installed in a Windows Server 2012.
I started the Spark master and tested the Spark Shell.
Then I configured the following in the conf\zeppeling-env.cmd file:
set SPARK_HOME=C:\spark-2.0.1-bin-hadoop2.7
set MASTER=spark://100.79.240.26:7077
I have not set the HADOOP_CONF_DIR and SPARK_SUBMIT_OPTIONS (that is optional according to the documentation)
I checked the values in the Interpreter configuration page and the spark master is Ok.
When I run the Zeppelin tutorial --> "Load data into table" note I am getting a connection refused error. Here is part of the messages in the error log:
INFO [2016-11-17 21:58:12,518] ({pool-1-thread-11} Paragraph.java[jobRun]:252) - run paragraph 20150210-015259_1403135953 using null org.apache.zeppelin.interpreter.LazyOpenInterpreter#8bbfd7
INFO [2016-11-17 21:58:12,518] ({pool-1-thread-11} RemoteInterpreterProcess.java[reference]:148) - Run interpreter process [C:\zeppelin-0.6.2-bin-all\bin\interpreter.cmd, -d, C:\zeppelin-0.6.2-bin-all\interpreter\spark, -p, 50163, -l, C:\zeppelin-0.6.2-bin-all/local-repo/2C3FBS414]
INFO [2016-11-17 21:58:12,614] ({Exec Default Executor} RemoteInterpreterProcess.java[onProcessFailed]:288) - Interpreter process failed {}
org.apache.commons.exec.ExecuteException: Process exited with an error: 255 (Exit value: 255)
at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)
at org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:48)
at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:200)
at java.lang.Thread.run(Thread.java:745)
ERROR [2016-11-17 21:58:43,846] ({Thread-49} RemoteScheduler.java[getStatus]:255) - Can't get status information
org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused: connect
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:189)
at org.apache.zeppelin.scheduler.RemoteScheduler$JobStatusPoller.getStatus(RemoteScheduler.java:253)
at org.apache.zeppelin.scheduler.RemoteScheduler$JobStatusPoller.run(RemoteScheduler.java:211)
Caused by: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused: connect
at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
... 8 more
Caused by: java.net.ConnectException: Connection refused: connect
at java.net.DualStackPlainSocketImpl.connect0(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:79)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
... 9 more
ERROR [2016-11-17 21:58:43,846] ({pool-1-thread-11} Job.java[run]:189) - Job failed
org.apache.zeppelin.interpreter.InterpreterException: org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused: connect
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:165)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:328)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105)
at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:260)
at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:328)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
In the Zeppelin logs there is only one file for zeppelin, the interpreter is an external Spark installation, which is not logging any error because it is never reached by the interpreter process.
I read some suggestion about the max and min memory of the JVM but I could not fix it yet.
Any comment will be appreciated.
Paul

Connection error: Request did not complete within rpc_timeout when doing cqlsh

I have installed Cassandra v2.0.16 on CentOS release 6.6 (Final).
But when I try to do cqlsh on it, It gives the following error,
Connection error: Request did not complete within rpc_timeout
On My local machine, I am running ubuntu 12.04, it works fine, but on my production machine (User Managed VPS running Cent OS) its not working properly.
I went to the /var/log/cassandra/system.log
ERROR [main] 2015-07-02 04:42:12,973 CassandraDaemon.java (line 571) Exception encountered during startup
org.jboss.netty.channel.ChannelException: Failed to bind to: localhost/127.0.0.1:9042
at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272)
at org.apache.cassandra.transport.Server.run(Server.java:159)
at org.apache.cassandra.transport.Server.start(Server.java:110)
at org.apache.cassandra.service.CassandraDaemon.start(CassandraDaemon.java:489)
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:567)
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:643)
Caused by: java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:444)
at sun.nio.ch.Net.bind(Net.java:436)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.jboss.netty.channel.socket.nio.NioServerBoss$RegisterTask.run(NioServerBoss.java:193)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:366)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:290)
at org.jboss.netty.channel.socket.nio.NioServerBoss.run(NioServerBoss.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
I am just stuck here.
Please any help would be appreciated.
I am not getting how to solve this problem.
Note: I have reinstalled Cassandra 2 to 3 times. But stuck with same problem

org.apache.thrift.transport.TTransportException: javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?

Am trying to enable node-to-node and client-to-node ssl encrytion in Datastax dse 3.2.3. First I tried to enable node-to-node it worked fine and started with out any error and then I added the client-to-node encryption options in the cassandra.yaml file, then it started throwing errors to me
ERROR [Thrift:1] 2014-01-15 16:22:22,628 TNegotiatingServerTransport.java (line 524) Failed to open server transport.
org.apache.thrift.transport.TTransportException: javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129)
at com.datastax.bdp.transport.server.TPreviewableTransport.readUntilEof(TPreviewableTransport.java:79)
at com.datastax.bdp.transport.server.TPreviewableTransport.preview(TPreviewableTransport.java:55)
at com.datastax.bdp.transport.server.TNegotiatingServerTransport.open(TNegotiatingServerTransport.java:189)
at com.datastax.bdp.transport.server.TNegotiatingServerTransport$Factory.getTransport(TNegotiatingServerTransport.java:517)
at com.datastax.bdp.transport.server.TNegotiatingServerTransport$Factory.getTransport(TNegotiatingServerTransport.java:408)
at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:193)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Caused by: javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
at sun.security.ssl.InputRecord.handleUnknownRecord(InputRecord.java:671)
at sun.security.ssl.InputRecord.read(InputRecord.java:504)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:882)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
... 9 more
DEBUG [WRITE-/x.x.x.x] 2014-01-15 16:22:27,586 OutboundTcpConnection.java (line 294) attempting to connect to /x.x.x.x
DEBUG [WRITE-/x.x.x.x] 2014-01-15 16:22:28,508 OutboundTcpConnection.java (line 294) attempting to connect to /x.x.x.x
ERROR [Thrift:2] 2014-01-15 16:22:32,926 TNegotiatingServerTransport.java (line 524) Failed to open server transport.
org.apache.thrift.transport.TTransportException: javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129)
at com.datastax.bdp.transport.server.TPreviewableTransport.readUntilEof(TPreviewableTransport.java:79)
at com.datastax.bdp.transport.server.TPreviewableTransport.preview(TPreviewableTransport.java:55)
at com.datastax.bdp.transport.server.TNegotiatingServerTransport.open(TNegotiatingServerTransport.java:189)
at com.datastax.bdp.transport.server.TNegotiatingServerTransport$Factory.getTransport(TNegotiatingServerTransport.java:517)
at com.datastax.bdp.transport.server.TNegotiatingServerTransport$Factory.getTransport(TNegotiatingServerTransport.java:408)
at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:193)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Caused by: javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
at sun.security.ssl.InputRecord.handleUnknownRecord(InputRecord.java:671)
at sun.security.ssl.InputRecord.read(InputRecord.java:504)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:882)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
... 9 more
My cassandra.yaml file looks like this.
server_encryption_options:
internode_encryption: all
keystore: /path/to/.keystore
keystore_password: xxxxx
truststore: /path/to/.truststore
truststore_password: xxxxx
client_encryption_options:
enabled: true
keystore: /path/to/.keystore
keystore_password: xxxxx
truststore: /path/to/.truststore
truststore_password: xxxxx
protocol: ssl
store_type: JKS
cipher_suites: [TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA]
This error means that you have a client which is attempting to open an unencrypted connection.

Setting up inter-node encryption in Cassandra

I am new to Cassandra and looking to setup internode encryption in Cassandra 1.2.8.
I have successfully created a keypair for the keystore and truststore following the steps outlined here:
http://docs.oracle.com/javase/6/docs/technotes/guides/security/jsse/JSSERefGuide.html#CreateKeystore
In the Cassandra.yaml file, I have adjusted the server encryption options to the following:
server_encryption_options:
internode_encryption: all
keystore: conf/keystore
keystore_password: password
truststore: conf/truststore
truststore_password: password
However, when I start the Cassandra server, I receive the following error:
ERROR 18:49:20,883 Fatal configuration error
org.apache.cassandra.exceptions.ConfigurationException: Unable to create ssl socket
at org.apache.cassandra.net.MessagingService.getServerSocket(MessagingService.java:410)
at org.apache.cassandra.net.MessagingService.listen(MessagingService.java:390)
at org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:589)
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:554)
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:451)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:348)
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:447)
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:490)
Caused by: java.io.IOException: Error creating the initializing the SSL Context
at org.apache.cassandra.security.SSLFactory.createSSLContext(SSLFactory.java:124)
at org.apache.cassandra.security.SSLFactory.getServerSocket(SSLFactory.java:53)
at org.apache.cassandra.net.MessagingService.getServerSocket(MessagingService.java:406)
... 7 more
Caused by: java.io.FileNotFoundException: conf\truststore\dev (The system cannot find the path specified)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(Unknown Source)
at java.io.FileInputStream.<init>(Unknown Source)
at org.apache.cassandra.security.SSLFactory.createSSLContext(SSLFactory.java:105)
... 9 more
Unable to create ssl socket
Fatal configuration error; unable to start server. See log for stacktrace.
ERROR 18:49:20,887 Exception in thread Thread[StorageServiceShutdownHook,5,main]
java.lang.NullPointerException
at org.apache.cassandra.service.StorageService.stopRPCServer(StorageService.java:321)
at org.apache.cassandra.service.StorageService.shutdownClientServers(StorageService.java:370)
at org.apache.cassandra.service.StorageService.access$000(StorageService.java:88)
at org.apache.cassandra.service.StorageService$1.runMayThrow(StorageService.java:519)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at java.lang.Thread.run(Unknown Source)
Please note the server runs without issues if the server encryption options is set back to none. Any thoughts/guidance would be appreciated.
Read the exception carefully:
Caused by: java.io.FileNotFoundException: conf\truststore\dev
(The system cannot find the path specified)
You've created the key/trust stores but you haven't pointed cassandra to them. In cassandra.yaml you need to enable SSL but you also need to specify the path to these two files. E.g:
server_encryption_options:
internode_encryption: all
keystore: C:\some\location
keystore_password: password
truststore: C:\some\other\location
truststore_password: password
Also remember to supply the key/trust store passwords instead of the example in cassandra.yaml.

Exception during bulk loads do Cassandra 1.1

I'm getting exception while bulk loads with sstableloader. I'm using JDK 1.6.0_25 64bit, Ubuntu 12.04 server. Ipv6 is turned off. Network communication between hosts works correctly. I'm going crazy ;-(
Exception in thread "Streaming to /192.168.219.36:1" java.lang.RuntimeException: java.net.SocketException: Invalid argument or cannot assign requested address
at org.apache.cassandra.utils.FBUtilities.unchecked(FBUtilities.java:628)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.net.SocketException: Invalid argument or cannot assign requested address
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at java.net.Socket.connect(Socket.java:478)
at java.net.Socket.<init>(Socket.java:375)
at java.net.Socket.<init>(Socket.java:276)
at org.apache.cassandra.net.OutboundTcpConnectionPool.newSocket(OutboundTcpConnectionPool.java:96)
at org.apache.cassandra.streaming.FileStreamTask.connectAttempt(FileStreamTask.java:245)
at org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTask.java:91)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
All hosts runs Cassandra 1.1 (datastax edition). Ports 7000,7199,9160 opened. Any ideas ??
Why are you streaming to port 1? It should be the RPC_PORT and the default for that is 9160.
/192.168.219.36: 1
What params are you passing to the bulkloader?

Resources