Connect remote scylla db server shows error - cassandra

I have installed scylla-db in google cloud servers.
Steps i have followed:
sudo yum install epel-release
sudo curl -o /etc/yum.repos.d/scylla.repo -L http://repositories.scylladb.com/scylla/repo/a2a0ba89d456770dfdc1cd70325e3291/centos/scylladb-2.0.repo
sudo yum install scylla
sudo scylla_setup
(Given yes to "verify supportable version" , " verify packages" , "core dump", " fstim ssd "
For remaining : Given NO)
IN file :/etc/scylla.d/io.conf
SEASTAR_IO="--max-io-requests=12 --num-io-queues=1"
( edited this file manually )
sudo systemctl start scylla-server
It shows: Cannot able to read yaml file. Then google it and downgraded the yaml-cpp version to 0.5.1 from 0.5.3 version.
then
scylla-server started running .
[root#scylla ~]# nodetool status
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns Host ID Rack
UN 127.0.0.1 208.69 KB 256 ? 888e91da-9385-4c61-8417-dd59c1a979b8 rack1
Note: Non-system keyspaces don't have the same replication settings, effective ownership information is meaningless
[root#scylla ~]# cat /etc/scylla/scylla.yaml | grep seeds:
- seeds: "127.0.0.1"
[root#scylla ~]# cat /etc/scylla/scylla.yaml | grep rpc_address:
rpc_address: localhost
#broadcast_rpc_address:
[root#scylla ~]# cat /etc/scylla/scylla.yaml | grep listen_address:
listen_address: localhost
[root#scylla ~]# cqlsh
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 3.0.8 | CQL spec 3.3.1 | Native protocol v4]
Use HELP for help.
cqlsh> exit
[root#scylla ~]# netstat -tupln | grep LISTEN
tcp 0 0 127.0.0.1:10000 0.0.0.0:* LISTEN 6387/scylla
tcp 0 0 127.0.0.1:9042 0.0.0.0:* LISTEN 6387/scylla
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1105/sshd
tcp 0 0 127.0.0.1:7000 0.0.0.0:* LISTEN 6387/scylla
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1119/master
tcp 0 0 0.0.0.0:9180 0.0.0.0:* LISTEN 6387/scylla
tcp 0 0 127.0.0.1:9160 0.0.0.0:* LISTEN 6387/scylla
tcp6 0 0 :::80 :::* LISTEN 5217/httpd
tcp6 0 0 :::22 :::* LISTEN 1105/sshd
tcp6 0 0 :::35063 :::* LISTEN 6412/scylla-jmx
tcp6 0 0 ::1:25 :::* LISTEN 1119/master
tcp6 0 0 127.0.0.1:7199 :::* LISTEN 6412/scylla-jmx
scylla-server is running.
Same setup was done another server
Server Name scylla-db-1
I need to connect to the server scylla ( IP: xx.xx.xxx) from this server.
when i execute the below :
[root#scylla-db-1 ~]# cqlsh xx.xx.xxx
Connection error: ('Unable to connect to any servers', {'xx.xx.xxx': error(111, "Tried connecting to [('xx.xx.xxx', 9042)]. Last error: Connection refused")})
How to connect the remote server from this server?
Also
while checking the http://xx.xx.xxx:10000 and http://xx.xx.xxx:10000/ui in the browser , I m getting problem loading page.
Note :
I have done editing the /etc/scylla.d/io.conf file for assigning the
max-io-requests manually

Port 10000 is the rest api for scylla and is usually left bounded to the 127.0.0.1 - thats why you can not access it
To gain access via cql you need to edit the /etc/scylla/scylla.yaml file and set the rpc_address
Please follow the instructions for configuring scylla for a cluster deployment: single dc http://docs.scylladb.com/procedures/create_cluster/ or multi dc http://docs.scylladb.com/procedures/create_cluster_multidc/.

You need set rpc_address on scylla.yaml and while connecting through cql give your rpc_address as well cqlsh xx.xxx.xxx.xx with user/pass if enabled.

Related

Cassandra running cqlsh fails

I am running Cassandra 2.1.15 on a virtual machine. The problem is that I cannot run the command "cqlsh" to create tables and keyspaces. It always gives me a timeout error after 2 minutes. I want to increase the timeout of cassandra. The virtual machine has 8Gb memory and 4CPUs. These are the list of things I have tried.
Creating ~/.cassandra/cqlshrc file and increasing client_timeout to 3600
Modifying the /usr/bin/cqlsh.py file and increasing DEFAULT_CONNECT_TIMEOUT_SECONDS and DEFAULT_REQUEST_TIMEOUT_SECONDS variables to 3600.
passing the argument --request-timeout 3600 --connect-timeout 3600 in cqlsh.
Using the IP in the listen address of cassandra.yaml file instead of localhost
I am aware that the performance of my virtual machine is very slow as it takes about 7 minutes to see the output of nodetool status command.
Nodetool status output:
Datacenter: site1
=================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns (effective) Host ID Rack
UN 192.168.1.6 196.08 KB 256 100.0% c88cf8ec-f7e1-41e1-aad0-85742c8c3ffb RAC1
Also I can see no error logs in the system.log file and from netstat command I can see that the server is listening for requests on ports 9042 and 9160
tcp6 0 0 :::9160 :::* LISTEN 32694/java
tcp6 0 0 :::11311 :::* LISTEN 2237/astaire
tcp6 0 0 :::9042 :::* LISTEN 32694/java
tcp6 0 0 192.168.1.6:7000 :::* LISTEN 32694/java
tcp6 0 0 127.0.0.1:40895 :::* LISTEN 32694/java
tcp6 0 0 127.0.0.1:7199 :::* LISTEN 32694/java
Kindly help me out.
A 8GB+4CPU spec for C* VM is good, it should not create any bottleneck or the lag you are experiencing. One think I can think of is you HDD not having enough IO bandwidth. Try checking Disk usage in Task Manager. (I assume you are using windows as you are running a VM for C*).
There are two alternatives that you can try if disk usage is the problem.
Switch to Docker. Docker is a lightweight VM comapred to the CentOS/ Ubuntu VM you are running. Docker quickstart is here and Dockerfiles are here.
Try C* on windows directly, using CCM if possible.

Cassandra linux cannot connect from remote

I have installed cassandra but not able to connect to cassandra server from remote ip..
[root#li1632-39 ~]# cassandra -v
3.0.9
I and connecting public_ip:9042 but connection refused. When I try to validate by telnet I can see port is closed.
When I try to check the status of cassandra its running.
[root#li1636-25 ~]# nodetool status
Datacenter: singapore
=====================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns (effective) Host ID Rack
UN 192.168.130.104 297.48 KB 256 100.0% 85bebb4d-4ce9-4144-b33a-8e9759a87e54 rack5
UN 192.168.130.59 262.73 KB 256 100.0% f79f1c04-b567-4e15-98f0-5fd1a8345f61 rack5
I have cassandra.yaml
listen_address: 192.168.130.59
rpc_address: 192.168.130.59
start_rpc: true
I have also tried with cassandra.yaml
listen_address: 0.0.0.0
rpc_address: 0.0.0.0
start_rpc: true
In this case I am getting below error.
[root#li1636-25 ~]# nodetool status
nodetool: Failed to connect to '127.0.0.1:7199' - ConnectException: 'Connection refused (Connection refused)'.
Error from the remote host on telnet.
A-MacBook-Air:~ ads$ telnet public_ip 9042
Trying 172.104.52.39...
telnet: connect to address public_ip: Connection refused
telnet: Unable to connect to remote host
Below is the result of Netstat
netstat -tulpn | grep LISTEN
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 3575/sshd
tcp 0 0 192.168.130.59:7000 0.0.0.0:* LISTEN 6777/java
tcp 0 0 127.0.0.1:7199 0.0.0.0:* LISTEN 6777/java
tcp 0 0 127.0.0.1:37857 0.0.0.0:* LISTEN 6777/java
tcp 0 0 192.168.130.59:9160 0.0.0.0:* LISTEN 6777/java
tcp6 0 0 192.168.130.59:9042 :::* LISTEN 6777/java
tcp6 0 0 :::22 :::* LISTEN 3575/sshd
I have stopped Firewalld also.

How to configure Cassandra on GCE then I can access it from my local java application running on Win7?

I created a CentOS on GCE and installed dsc-cassandra 3.0. Then I changed the rpc_address from localhost to the internal ip or external ip in cassandra.yaml.
On the VM, I started cassandra and use cqlsh to access cassandra successfully. But I couldn't use cqlsh internal_ip or cqlsh external_ip.
Also, I turned on tcp:9042 port for this instance.
But I still couldn't access Cassandra from my local java app with the NoHostAvailableException(Cannot connect).
By the way, I did the same thing of my local VM running with VM VistualBox. I could access it.
Running sudo netstat -lntp | grep pid displayed:
tcp 0 0 127.0.0.1:33743 0.0.0.0:* LISTEN 1207/java
tcp 0 0 127.0.0.1:7000 0.0.0.0:* LISTEN 1207/java
tcp 0 0 127.0.0.1:7199 0.0.0.0:* LISTEN 1207/java
tcp6 0 0 127.0.0.1:9042 :::* LISTEN 1207/java
The Ip address was still 127.0.0.1. I think this is the problem.
How to configure the cassandra.yaml file?
I know where I was wrong.
I used sudo service cassandra restart to restart cassandra after editing the cassandra.yaml. The terminal showed:
Restarting cassandra (via systemctl): [ OK ]
Actually, I think it didn't really restart it. Then I used nodetool stopdaemon to stop cassandra and then start it again. The configuration of cassandra.yaml worked.
Helpful commands:
1.
ps aux | grep cassandra
sudo netstat -lntp | grep <cassandra_pid>
Using these commands to verify the ip/port of the cassandra service on remote VM.
tcp 0 0 127.0.0.1:7000 0.0.0.0:* LISTEN 5928/java
tcp 0 0 127.0.0.1:42682 0.0.0.0:* LISTEN 5928/java
tcp 0 0 127.0.0.1:7199 0.0.0.0:* LISTEN 5928/java
tcp6 0 0 10.138.0.2:9042 :::* LISTEN 5928/java
2.
telnet <cassandra_ip> 9042
Using this command to verify the ip/port of the cassandra service on local machine.

Restricting Cassandra to localhost only

I installed cassandra as a service on Ubuntu. Test Cluster is accessible on 127.0.0.1:9042. I want to restrict everything related to cassandra to localhost only, nothing open to internet. Currently, this is what I see on netstat -tulpen:
udp 0 0 130.159.223.50:123 0.0.0.0:*
udp 0 0 0.0.0.0:123 0.0.0.0:*
udp6 0 0 fe80::215:5dff:fcdf:123 :::*
udp6 0 0 ::1:123 :::*
udp6 0 0 :::123 :::*
Disclaimer: What you want to achieve can be done through configuration, but for a production cluster, it should be done using a firewall.
You need to modify a number of settings in cassanra.yaml to listen only for the loopback address (127.0.0.1)
listen_address: 127.0.0.1
rpc_address: 127.0.0.1
# make sure the broadcast address is commented out
# broadcast_address: 1.2.3.4
When running nodetool you should see the node's ip as the loopback interface ip
$ nodetool status
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns Host ID Rack
UN 127.0.0.1 60.62 TB 256 ? e7060cda-f99b-495c-ad55-2d380b4d452e rack1
Note: Non-system keyspaces don't have the same replication settings, effective ownership
information is meaningless
You can then verify that cassandra is innacessable over the public or private ip but only on the loopback ip with telnet:
core:cassandra core$ telnet <external ip> 9042
Trying 134.103.x.x...
telnet: connect to address 134.103.x.x: Connection refused
telnet: Unable to connect to remote host
core:cassandra core$ telnet <internal ip> 9042
Trying 10.17.x.x...
telnet: connect to address 10.17.x.x: Connection refused
telnet: Unable to connect to remote host
core:cassandra core$ telnet 127.0.0.1 9042
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
^]
telnet> Connection closed.

Linux Centos 6 Some how filtered port 80 + broke Apache 2

I have magically been able to break port 80 / Apache server when following a guide to install PHPMyAdmin (http://www.krizna.com/centos/installing-apache2-mysql-server-php-centos-6-lamp/#apache)
Prior to me to starting the guide, Apache 2 was working for me (was able to view pages and see the default blue one)
I followed the guide down to Step 4 under "PHP installation" and went to check the page and I was getting the following error (note: I skipped Mysql installation since I have it already installed and "Testing your page" in the Apache section since I had it working prior).
ERROR
The requested URL could not be retrieved
The following error was encountered while trying to retrieve the URL: http://`192.168.141.22`/
Read Error
The system returned: (104) Connection reset by peer
An error condition occurred while reading data from the network. Please retry your request.
Your cache administrator is webmaster.
Generated Wed, 03 Jul 2013 19:04:56 GMT by tx22rrpep4da (hpm/3.0.55)
I tried to uninstall (yum erase) httpd and php and re-install and no success.
I did an nmap and it shows 80/tcp filtered http.
I even disabled/stopped IP tables incase that was the issue, no luck. I have other things like TS3 running on other ports fine.
Restarted the dedicated server.
netstat -tulpn
te PID/Program name
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 25896/httpd
tcp 0 0 0.0.0.0:30033 0.0.0.0:* LISTEN 2967/./ts3server_li
tcp 0 0 127.0.0.1:53 0.0.0.0:* LISTEN 2736/named
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 2798/sshd
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 2873/master
tcp 0 0 127.0.0.1:953 0.0.0.0:* LISTEN 2736/named
tcp 0 0 0.0.0.0:10011 0.0.0.0:* LISTEN 2967/./ts3server_li
tcp 0 0 ::1:53 :::* LISTEN 2736/named
tcp 0 0 :::22 :::* LISTEN 2798/sshd
tcp 0 0 ::1:953 :::* LISTEN 2736/named
udp 0 0 0.0.0.0:9987 0.0.0.0:* 2967/./ts3server_li
udp 0 0 127.0.0.1:53 0.0.0.0:* 2736/named
udp 0 0 ::1:53 :::* 2736/named
I had a few other people test the IP in case it was just my connection but they all got the same as well (IP: 192.168.141.22 )
Really hoping I can get this fixed with out doing something over the top like wiping and re-installing centos 6.
The netstat command is showing that Apache is in-deed listening on the following IP & port:
0 0.0.0.0:80
If you want reach Apache in 192... ip, then you need to edit one of the .conf files that has the "Listen" directive(normally in /etc/httpd/conf.d/ports.conf or the main .conf file) and either put in the ip or remove all ips so Apache can listen on all interfaces available in the machine.

Resources