I am running a stand-alone Spark 3.2.1 locally, on my mac, installed via brew. This is for low-cost (free) unit testing purposes. I am starting this instance via pyspark command from terminal and able to access the instance web ui.
I am also trying to run spark-submit locally (from the same mac) to run a pyspark script on the pyspark instance described above. When specifying the --master :7077 I am getting the "connection refused" error. It does not look like the port 7077 is open on my mac.
How do I open the port 7077 on my mac such that I can access it from my mac via spark-submit, but other machines on the same network cannot?
Could someone share clear steps with explanations?
Much appreciated :)
Michael
Check your spark master process is running.
It must be like following output.
jps
$PID Master
$PID Worker
If spark process is not running,
run script $SPARK_HOME/sbin/start-master.sh in your shell first.
also $SPARK_HOME/sbin/start-worker.sh.
and then check if process listen on 7077 port with following command.
sudo lsof -nP -i:7077 | grep LISTEN
I have installed cassandra 3.11.4 as per the below link http://cassandra.apache.org/download/ and when I'm trying to run cqlsh and nodetool command its showing command not found.
when run sudo service cassandra status,cassandra is running.
Please help
First, you need to verify that the service is healthy and operational
verify that the process is running:
ps -efa | grep cassandra
The expected outcome is a valid java process being executed
verify the log file as defined in $CASSANDRA_HOME/conf/logback.xml
tail -n 100 cassandra/system.log
The expected outcome is to not find errors
Try to replicate the issue in a new instance of the command line
You need to verify the services whether it is running or not in the configured port and IPs.
Also, you can check the system.log for any error. if still issue and data is not important you can delete the /data /commitlog /savedcache and restart the cassandra services again.
once started just check
ps -ef | grep cassandra
I installed Cassandra on OSX.
When trying to run cqlsh from /bin/, I get the following error:
> Connection error: ('Unable to connect to any servers', {'127.0.0.1':
> ConnectionShutdown('Connection <AsyncoreConnection(4522252560)
> 127.0.0.1:9160 (closed)> is already closed',)})
I had difficulty installing Cassandra, and had used brew install cassandra, installed DSE from datastax, and downloaded a binary from Apache.
I think the issue was the result of having a previous installation that wasn't properly uninstalled. My user .cassandra directory had some default configs, that must have been for a different version, and prevented cqlsh from starting.
rm -Rf ~/.cassandra
After that, I could run cqlsh properly.
$ bin/cqlsh
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 2.1.2 | CQL spec 3.2.0 | Native protocol v3]
Use HELP for help.
cqlsh>
Running cassandra helped me to troubleshot the problem:
> cassandra
Cassandra 3.0 and later require Java 8u40 or later.
and:
> java -version
java version "1.7.0_95"
OpenJDK Runtime Environment (IcedTea 2.6.4) (7u95-2.6.4-3)
OpenJDK 64-Bit Server VM (build 24.95-b01, mixed mode)
After installing Java 8 it worked.
Cassandra-2.1 uses 9042 as cqlsh port.Connect on this it is default port.If you want to connect on 9160 port then change it by setting environment variable CQLSH_PORT and CQLSH_HOST but for setting this you have to change rpc_port from default 9160 to any other port.
Well, I faced same issue and this is what i did on Mac.
$cd /usr/local/Cellar/cassandra/3.10_1/bin/
Or wherever your cassandra is installed.
$cassandra
this initializes all configuration in your local system to start cassandra.
$./cqlsh
This lets you enter cql query mode.
I resolved this in DSE-5.0 by editing below file:
vi ~/.cassandra/cqlshrc
And editing below configs to current IP address (192.168.1.6) of the machine I am running. In my case it's my MacBook.
[connection]
hostname = 192.168.1.6
port = 9042
; factory = cqlshlib.kerberos_ssl.kerberos_ssl_transport_factory
; [kerberos]
; hostname = 192.168.1.6
; service = cassandra
; optional
; principal = bill/cassandra-admin#foo.example.com
I hope it will help someone.
Thanks,
Chandan
I faced this issue when I was trying to connect to Cassandra 2.0.1 version using cqlsh 5.0.1
[cqlsh 4.1.1 | Cassandra 2.0.11.83 | CQL spec 3.1.1 | ]
[cqlsh 5.0.1 | Cassandra 2.1.2 | CQL spec 3.2.0 ]
This is just one of the reasons for the error "Connection error: ('Unable to connect to any servers',".
When I used the cqlsh client compatible with Cassandra 2.0.1, I was able to connect to the cluster.
Hope this helps to find the root cause.
I got the same error after moving from Java 8 to Java 11. From https://github.com/nosan/embedded-cassandra/issues/21, Apache Cassandra does not yet have support for java 11.
Reverting to the system's current java version to Java 8 solved the problem.
Just add/modify the following into the cassandra.yaml:
rpc_address: 127.0.0.1
seeds: "127.0.0.1"
listen_address: 127.0.0.1
Note that, the values in all these fields must be same.
That means, if you are pointing to a static IP, then the same IP should be used. And if you are just pointing the localhost, then localhost must be used in all fields.
I got myself into a mess so here was my scenario if anyone else runs into it:
I followed this tutorial
http://www.datastax.com/2012/01/working-with-apache-cassandra-on-mac-os-x
(which is somewhat outdated).
I got everything installed and working - all good. Then I decided to download Datastax Enterprise tools to try them out
Ran into errors during the install but the install went through. Again following the tutorial above, tried running the ./datastax-agent program and it failed. Decided to just bag it so ran their uninstaller in their installation folder.
Tried to go back to original cassandra folder (what I had working in step 2) and run "sudo ./cassandra" then I started getting JMX error (message "Unable to bind JMX, is Cassandra already running?") which led me to this link about changing the port which i did (not really fixing the issue but working around it) run two cassandra versions in the same machine.
Got the server service running underneath a new port but when trying to connect with the client "sudo ./cqlsh" would not work (would get "Error=none" or destination not found).
Did the remove step #Peter mentioned above and tried again and still did not work.
Decided to go back and see what was holding onto the original port cassandra was using which I changed in step 4 Who is listening on a given TCP port on Mac OS X?
Determined that JAVA.app was hanging onto it so killed it within Activity Monitor.
Reran the remove command #Peter mentioned above again and decided to download the stable release (v2.0.13) instead of the dev release and walk through the tutorial from step 1 again.
Exited and re-opened terminal. Ran "sudo ./cassandra" - worked perfect running under standard ports. Open another terminal window and ran "sudo ./cqlsh" and this also worked correctly.
I believe between Java hanging onto the default port 9042 (maybe from the DataStax install errors), version/host/port settings within ~/.cassandra/cqlshrc and using the dev version of cassandra these were all causing me fits.
I was having trouble connecting Cassandra from the terminal
[
and then I tried to connect using /usr/local/Cellar/
[
and then invoke cassandra. Then opened a new terminal and typed cqlsh the Cassandra started.
Hope this may also applicable one to resolve Cassandra issue.
install java jdk and Set JAVA_HOME in Environment Variables or set JAVA_HOME = java path in command prompt.
after
run cassandra.bat
it will show the logs if you get any error
You need to start Cassandra form the install_location's bin directory.
For example, this command will start Cassandra in the foreground:
$ ./cassandra -f
Then open another terminal tab (command + t) and run cqlsh
$ ./cqlsh
Ensure that DEFAULT_HOST = '127.0.0.1' in script cqlsh.py is pointing to the node IP where you have done your cassandra installation.
In my case I had installed cassandra on 10.160.1.5, edited above file and it started working.
Running cqlsh gets this error for me but running bin/cqlsh from the cassandra directory works.
I faced the same issue. I ran cassandra.bat --> This appears to have started the server. Then I opened a new command prompt (run as an administrator), changed directory to the cassandra/bin location. When I ran the command cqlsh, voila it worked!
I solve this issue by just simply restarting cassandra sudo systemctl restart cassandra
In case of MacOSX, it worked to just run
cassandra
rather than running it through brew services start cassandra.
I was having the same problem with Cassandra configured with SSL.
cqlsh --ssl
Solves connects correctly.
I installed single node cluster in my local dev box which is running Windows 7 and it was working fine. Due to some reason, I need to restart my desktop and then after that whenever I am doing like this on the command prompt, it always gives me the below exception-
S:\Apache Cassandra\apache-cassandra-1.2.3\bin>cassandra -f
Starting Cassandra Server
Error: Exception thrown by the agent : java.rmi.server.ExportException: Port already in use: 7199; nested exception is:
java.net.BindException: Address already in use: JVM_Bind
Meaning port being used somewhere. I have made some changes in cassandra.yaml file so I need to shutdown the Cassandra server and then restart it again.
Can anybody help me with this?
Thanks for the help.
in windows7, with apache cassandra, a pid.txt file gets created at the root folder of cassandra. Give following instruction to stop the server:
d:/cassandra/bin> stop-server -p ../pid.txt -f
Running -f starts the server as a service, you can stop it through the task manager.
It sounds like your Cassandra server starts on it's own as a service in the background when your machine boots. You can configure windows startup services. To run cassandra in the foreground on windows simply use:
> cassandra.bat
If your are using Cassandra bundled with DataStax Community Edition and running as a service on startup of your machine then you can execute following commands to start and stop Cassandra server.
Start command prompt with admin rights
run following commands
net start DataStax_Cassandra_Community_Server
net stop DataStax_Cassandra_Community_Server
My Cassandra used to work with no problems.
I was able to connect with no problems but now for some reason it doesn't work anymore.
[default#unknown] connect localhost/9160;
Exception connecting to localhost/9160. Reason: Connection refused.
I am in Ubuntu server
Thanks in-advance
The solution to this question was provided to you on the pycassa google group:
https://groups.google.com/d/topic/pycassa-discuss/Bai7bvkHYU4/discussion
This is not a pycassa problem. The problems you are having are specific to starting a Cassandra instance and not following the documentation in the README.txt that is in the root folder of the distribution:
Getting started
This short guide will walk you through getting a basic one node cluster up
and running, and demonstrate some simple reads and writes.
tar -zxvf apache-cassandra-$VERSION.tar.gz
cd apache-cassandra-$VERSION
sudo mkdir -p /var/log/cassandra
sudo chown -R whoami /var/log/cassandra
sudo mkdir -p /var/lib/cassandra
sudo chown -R whoami /var/lib/cassandra
Note: The sample configuration files in conf/ determine the file-system
locations Cassandra uses for logging and data storage. You are free to
change these to suit your own environment and adjust the path names
used here accordingly.
Now that we're ready, let's start it up!
bin/cassandra -f
Running the startup script with the -f argument will cause Cassandra to
remain in the foreground and log to standard out.
Now let's try to read and write some data using the command line client.
bin/cassandra-cli --host localhost
The command line client is interactive so if everything worked you should
be sitting in front of a prompt...
Connected to: "Test Cluster" on localhost/9160
Welcome to cassandra CLI.
Type 'help;' or '?' for help. Type 'quit;' or 'exit;' to quit.
[default#unknown]
As the banner says, you can use 'help;' or '?' to see what the CLI has to
offer, and 'quit;' or 'exit;' when you've had enough fun.
Verify the following:
Cassandra process is running and thrift is listening on 9160 (netstat-tulpn)
9160 port not being blocked by a firewall rule or similar
If the above are true, then check the cassandra log for additional information.
Other than that your description is pretty vague. So any other information about what may have changed in the environment would be helpful.
I faced the same problem and the reason was that I had configured Cassandra to listen on the Server IP and not on localhost.
/etc/dse/cassandra/cassandra.yaml
listen_address: 10.102.8.71
So try this and check if it works for you:
cassandra-cli --host "your host name"