To find out some info about a local spark process, launched locally via spark-shell command I can do:
jps -lm | grep -i spark
However, how do I find what TCP port the UI is published on?
I have tried:
lsof -p PID
but no luck.
If you list open ports, you can filter by greping the PID (My pid is 30688, but you can also grep by java or something just to narrow the results):
$ ss -l -p -n | grep -i 30688
And, unless you've reconfigured it to a dramatically different range, you should be able to see a 40* port (I replaced tabs with -- to save space):
tcp--LISTEN--0--50--*:4040--*:*--users:(("java",pid=30688,fd=275))
Seems you already have the PID. in that case you can run below, which should indicate what ports the process is listening on:
netstat -tunlp|grep LISTEN|grep PID
The spark driver process may be listening on more than one ports, so you may have to try http://host:port for the ports obtained.
Alternatively, if you are finding only one PIDs from your jps | grep, then you can be sure that the port is 4040(default spark web ui port), unless you find the args spark.webui.port. In the latter case, get the port from the args value.
Related
I am not able to find a port number for running Tomcat server on Ubuntu
i.e. netstart -anp | grep 'tomcat' but not getting any output.
You will want to grep for java and not tomcat, as the process binary is java and not Tomcat.
If you'd like to find the PID for the process to make things easier (e.g. if you have lots of Java processes on the server), you can do this:
ps aux | grep catalina
This will show you your various Tomcat processes. Each one will have a system property on the command-line like this:
-Dcatalina.base=/path/to/your/tomcat
Note that there is also catalina.home which may be different. If they are different, it is catalina.base which is the correct one, which contains your server's conf/server.xml which controls the server.
Once you have that PID, you can netstat -plan | grep [PID] to get your port number.
Or just look in /path/to/your/tomcat/conf/server.xml for any <Connector> elements, each of which should have a port specified.
I have a problem when launching a Nodejs script that listens in one of the HTTP ports. Sometimes, even if I stop the script, the used HTTP port stays "in use", making it impossible to use it another time. Today, i've set up NGINX in my linux and all the HTTP ports were "in use". I was obliged to restart my computer to solve the problem.
I wanted to know why is this happening ? What can i do to prevent it ? and in case an HTTP port stays "in use", how can i close it to be able to use again ?
Thanks for your help.
This is applicable only on Linux and MacOS, you can list all your used ports like that:
sudo lsof -i -P -n | grep LISTEN
Read more here about how to check if a port is in use: https://www.cyberciti.biz/faq/unix-linux-check-if-port-is-in-use-command/
You can also list the node processes:
top | grep node
or
ps -ef | grep node
Then you can kill the node processes like that:
killall node
Make sure that when you want to stop the server you are pressing CTRL + C
I encounted a problem today:
When I started HDP docker container, an error occured:
listen tcp 0.0.0.0:8086: bind: address already in use
According to error message, I know that port 8086 was already in use, so I tried some commands to determine which program was using port 8086.
lsof -i:8086
lsof -i tcp:8086
lsof | grep 8086
But all of commands above make no outputs!
I felt really confused about that, after some searching on google, I tried another command:
netstat -pna | grep 8086
I got correct output from this command.
I know some differences between lsof and netstat, but I really do not know why I cannot get any output from lsof -i:8086?.
Here are some differences between two commands I searched from google:
netstat(net statistic) is connection based,it shows NW connections (udp/tcp ports), routing tables, interface, multi-cast membership, etc.
lsof(list of open files) is application based, this is kind of like netstat + ps, there you can see all accessed ports, NW connections, etc.
but lsof includes stuff like my local emacs window terminal session (tty dev/pts/n) which is not part of netstat
I faced a similar issue today. The solution was to run the lsof command with sudo privileges.
sudo lsof -i:8086
should print the desired output.
LSOF: List of Open Files. It lists all the open files belonging to all active processes.
Examples:
sudo lsof -n -i
sudo lsof -n -i4
sudo lsof -n -i :80
-n inhibits the conversion of network numbers to host names for network files. Inhibiting conversion may make lsof run faster. It is also useful when host
lookup is not working properly
-i selects the listing of files any of whose Internet address matches the address specified in i. If no address is specified, this option selects the listing of all Internet and x.25 (HP-UX) network files. If -i4 or -i6 is specified with no following address, only files of the indicated IP version, IPv4 or IPv6, are displayed.
NETSTAT: It is a tool to get the network statistics. By default, netstat displays a list of open sockets. If you don't specify any
address families, then the active sockets of all configured address
families will be printed.
Displays the kernel routing tables:
netstat -r
Display all listening and established connection for both TCP and UDP with PID data:
netstat -plunt
Additionally, You have another command line tool to use which is SS.
SS: It is used to dump socket statistics. It allows showing information similar to netstat. It can display more TCP and state
information than other tools.
-plunt gives data for the TCP and UDP connections which are established and listening with process information:
sudo ss -plunt
You should be root to get proper answers to your lsof questions. Your command is fine, assuming something really is listening on that port.
As you already mentioned, lsof is a very useful command which is used to list files opened by a specific process, while netstat is a tool for monitoring network connections.
You should be able to find the PID of the process listening on port 8086 with netstat:
netstat -tunlp |grep :8086
and then use lsof to list the files used by the process:
lsof -p PID
I am new to the Cassandra database. I have downloaded Cassandra and set the JAVA_HOME. When I try to run, the following exception is thrown:
Exception thrown by the agent : java.rmi.server.ExportException: Port already in use: 7199; nested exception is:
java.net.BindException: Address already in use: JVM_Bind.
Can anyone assist me with this issue?
For those who are facing the same problem using Ubuntu OS a solution can be to kill CassandraDaemon :
pkill -f CassandraDaemon
When you launch for the first time the command "cassandra -f" background daemon is running so Ctrl^C doesn't stop the process.
You could check if cassandra is running by checking the port
lsof -i :9160
if you got a result back that means it is running
If you want to kill it , do kill -9 "then the pid you got from the last step"
if you want to see the ongoing log run
cassandra -f when you start cassandra
I appears that Cassandra is already running in the background. Try connecting using cassandra-cli.
ps -ax | grep cassandra, take note of the process id
kill <pid>
sudo ./cassandra
Port 7199 is the default Cassandra's JMX port( used for monitoring).
In case you are trying to run multiple instances on one physical machine, modify $CASSANDRA_HOME/conf/cassandra-env.sh configuration file and set different port, for example 7299
JMX_PORT="7299"
I am writing the same but for windows developer command prompt :
Let say its showing issue with 9042 port
netstat -ano | findstr : 9042
List all the process using port 9042
taskkill /PID 237979 /F
here 237979 is the processid which is using port 9042
Download the TCPView from http://technet.microsoft.com/en-us/sysinternals/bb897437
Open the TCPView application and sort the output by Port
Click on the record which points to Port - 7199
Right click and "End Process"
Now, run the Cassandra.bat and it should work.
I am running CentOS 5 with csf firewall. I'm running a program that can't connect to another server (using some port that is blocked by csf I presume). Where is the log file for 'ports'?
Netstat is the command to use to get ports and network activity. To diagonise server processes I usually use:
netstat -tln
This yields port numbers in tcp mode listening. To identify associated processes you can also use -p to grab the pid. Here is the IANA ports list.
I found my answer right after searching a few more threads.
# tail -f /var/log/messages
Shows the UDP message but not the port.... Hmm....