I'm using netcat as a bridge between some services and a spark streaming instance. For example, the service sends message to an host:port that net cat is listening to and then the idea is that spark can then consume this. However is there a way to make netcat passthrough, i.e. to act as a really simple server and literally listen and emit.
ncat -lk localhost 5005
This shows that I can get my messages send from my first service. But I get nothing from spark which listens to the same host:port. Is there a way to make this work?
One suggestion was to use the piping with mkfifo backpipe however the problem now is that when running my spark instance is listening on 5006 but it seems this connection doesn`t seem alive. My service sends to 5005 and then netcat should pipe it to 5006, but how can I make the service on 5006 to be always present such that my spark instance can listen to it?
mkfifo backpipe
nc -kl localhost 5005 0<backpipe | nc localhost 5006 1>backpipe
I also tried the following for good measure:
nc -klp 5005 -w 5 localhost 5006
But the issue is always that spark cannot consume with the following error:
Deregistered receiver for stream 0: Restarting receiver with delay 2000ms: Error connecting to 127.0.0.1:5006 - java.net.ConnectException: Connection refused
What is the version of netcat you use? And what operating system? If you use traditional netcat (not BSD-one used for Spark examples) you have to provide port argument:
nc -lk -p 5005
Related
I have a Linux Server running Redhat Rhel 7 and a Device called "Compoint Lan System (Colas)" (german manufacturer). The Colas has its own firmware so I don't know if it's based on linux. The Colas is set as a TCP-Client. It receives messages from its serial 1 port. I get the messages coming from the serial port 1 of the colas on my server with rsyslog.
Now what I want is to send a string (2 letters) from my server (tcp-server) to my colas's serial port 1 (tcp-client) to get information of the device attached to serial port 1.
Is there a command in linux to accomplish that? Something like "command 'string message' destination port"? I am sorry if it isn't written well.
Install netcat
yum install nc
Make it to listen to a particular port number
nc –l portnumber &
Lets validate it using netstat from a different console:
netstat -anlp |grep yourportnumber
PS: Change the installation command based on your linux flavor.
Ranadip Dutta's answer meets your requirement. The listen there doesn't mean listen for incoming data, it rather means listen for connection request from client. Of course you can't use rsyslog and nc as the server at the same time, but with nc you get the messages coming from the Colas displayed as well as the characters you enter sent.
When I'm trying to run a service (RavenDB) on port 8080 it stops and the Windows Logs show the following error:
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.Net.HttpListenerException: Failed to listen on prefix 'http://+:8080/' because it conflicts with an existing registration on the machine.
Acccording to IIS and netstat -an | find "8080" the port is currently not in use.
If I change the port to any other, the problem disappears.
Port 8080 may actually be in use. To replicate another answer
netstat -a only lists connected sockets and listening socket.
-a Displays all connections and listening ports. Neither
connect nor listen was called on your socket, so it falls outside the
purview of netstat -a.
However, since Windows 10, you can use netstat -q.
-q Displays all connections, listening ports, and bound
You could also try to view the port using tcpview from the SysInternals suite. Sort by port number. It will also tell you the process using the port, which you can then kill.
For testing purposes I want to build a server that listens for TCP connections to every port (or at least on most of the ports) on a certain interface (e.g., eth0). The server is accessed via SSH to eth1, so no problem there. (I do not care about UDP or other protocols)
I want to do a special kind of middlebox detection/analysis, therefore I need to be able to fully establish a connection. Having an HTTP connection would be the best, because the "client" could be implemented as JS in the Browser.
I started with a simple jetty server, but had to realize that jetty needs to spawn at least on thread per port it is listening to. This leads to problems when I want to listen to several thousand ports. Or is there a way around that?
My next try was to use iptables:
sudo iptables -t nat -A PREROUTING -i eth0 -p tcp -j DNAT --to-destination 127.0.0.1:8080`
It seemed to work. It allows to connect on every port and the traffic gets routed to the local port 8080 where jetty listens. But now I no longer know which port was used by the client. Because jetty thinks the connection was established via port 8080. Is there a way to determine the real incomming port from jetty? I could send the port as part of the HTTP request, but if the client tries to contact port 1234 .. and a middlebox redirects this to port 5678 .. I am unable to know what port was used.
I also tried userland solutions like socat. The problem was even worse than before. Because now jetty also saw the remote IP as being 127.0.0.1.
Or, is there another way to achieve this?
Oh and btw: I have full control of the machine. So I could change the kernel or whatever is needed. Right now I use Ubuntu 14.04 LTS, but if a solution needs something else I could go with that.
NB: This is a Python solution, because I know Python, but you can accomplish the same thing in any language the exposes the underlying C library getsockopt call.
If you replace your DNAT rule with a REDIRECT rule, you can then
use getsockopt with the SO_ORIGINAL_DST option to retrieve the
original address of a REDIRECT-ed connection.
Consider the following code:
#!/usr/bin/python
import socket
import struct
SO_ORIGINAL_DST = 80
s = socket.socket()
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(('0.0.0.0', 2000))
s.listen(10)
while True:
csock, caddr = s.accept()
orig_dst = csock.getsockopt(socket.SOL_IP, SO_ORIGINAL_DST, 16)
orig_port = struct.unpack('>H', orig_dst[2:4])
orig_addr = socket.inet_ntoa(orig_dst[4:8])
print 'connection from', caddr
print 'connection to', (orig_addr, orig_port)
print
If I have an iptables rule that looks like:
# iptables -t nat -A PREROUTING -p tcp --dport 1500:1600 \
-j REDIRECT --to-port 2000
And while the above Python code is running I connect from another
host to my_ip_address:1500, I see:
connection from ('192.168.1.20', 35790)
connection to ('192.168.1.75', (1500,))
And if I connect to port 1550 I see:
connection from ('192.168.1.20', 42054)
connection to ('192.168.1.75', (1550,))
Which I think is exactly what you were asking for. Note that to my knowledge this will only work for TCP connections; there are other solutions (possibly involving the TPROXY iptables target) that may work with UDP connections as well.
I have a RedHat node behind central firewall and I would like to be able to check if the firewall guys have opened a port for me on which still nothing in listening. When I want to check that for TCP port I use telnet and I get:
When not opened:
[myname#78 ~]$ telnet myhost 4080
Trying myhostip...
telnet: connect to address myhostip: Connection refused
telnet: Unable to connect to remote host: Connection refused
[myname#78 ~]$
When opened:
[myname#78 ~]$ telnet myhost 4080
Trying myhostip...
Connected to myhost (myhostip).
Escape character is '^]'.
So how do I do that for an UDP port?
You can accomplish this using Netcat which you can install on RHEL with yum(1) using command yum install nc.
After you've installed Netcat, put it in listen mode on the receiving end ie. on the host you're trying to reach from outside world. You can do this with nc -l -u <your external I -address> 4080. This command will hang on your terminal and wait for any connections to port you specify on the command line.
Then connect from outside world, using Netcat again, but this time leave the -l out ie. use netcat -u <your external IP address> 4080. This command will also just sit there. Typing any input now should be visible on the receiving end Netcat. In case it is not, something blocks the traffic between two the two hosts.
When i run the jboss-cli.sh,
I get this message.
[root bin]# sh jboss-cli.sh
You are disconnected at the moment. Type 'connect' to connect to the server or 'help' for the list of supported commands.
[disconnected /] connect localhost
The controller is not available at localhost:9999
[disconnected /] connect
The controller is not available at localhost:9999
[disconnected /] connect localhost:9999
The controller is not available at localhost:9999
[disconnected /]
Also i have another installation of jboss5 GA. I hope that is not interfering.
Although that is totally shut down for now.
Native management interface is :9999 in standalone.sh
Please throw light on this issue.
#
EDITED
#
When i stop my service with "service jboss stop"
i get this message
[root# bin]# *** JBossAS process (7302) received KILL signal ***
grep: /var/run/jboss-as/jboss-as-standalone.pid: No such file or directory
I Dont know how to check whether server is listening on the port 9999 or not.
Few more details
[root bin]# netstat -anp |grep 9999
tcp 0 0 127.0.0.1:9999 0.0.0.0:* LISTEN 7931/java
[root bin]# netstat -anp |grep 8080
tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN 7931/java
JBoss processs id and the server id acquiring these ports is same.
This question has two issues ,
First, i have provided debuging parameter in the startup script.
If you see 8787 that means you have somewhere provided debuging argument.
Second and the most important one controller not available #localhost or #IPADDRESS .
Please check if you have used port offset, as it increments all the ports by the number with with you have set port offset.
Suppose port offset is 2
Then try to access connect localhost:10001 Port i.e 9999+2
On my production server sometimes it does not works with localhost , but works with IP address.
Then try to access connect IPADDRESS:9999
OR
Then try to access connect 127.0.0.1:9999
Please check in the firewall weather the port 9999 or what ever with port offset, if the port is not open in the firewall it gives error,
I asked this question 6 months back and the above checks has solved
the problem always.
This is probaby because you have changed your binding configuration and jboss does not bind to 127.0.0.1.
In case your jboss instance is not binding to 127.0.0.1, you may use --controller option as follows:
./jboss-cli.sh --controller=YOUR_IP:9999
Use netstat -anp |grep 9999 to find out if port 9999 is in use and by which process id. You could also check the host.xml used by the controller to configure the proper native port.
In the host xml, you should find the default port:
<native-interface security-realm="ManagementRealm">
<socket interface="management" port="${jboss.management.native.port:9999}"/>
./jboss-cli.sh --controller=localhost:9999 --connect
You open the debug-port with jboss-cli.sh. Either you activated in jboss-cli.sh:
# Sample JPDA settings for remote socket debugging
# JAVA_OPTS="$JAVA_OPTS -Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n"
or you set JAVA_OPTS with such an option in you environment. See
echo $JAVA_OPTS
I guess you did this for two jboss-processes, and you get a port-conflict. See
netstat -nap | grep 8787
I recently faced this issue and the root cause that I found was completely different than it is listed above. It is because for some other project I shifted to JDK 1.8 from 1.7. Boom! and error started coming up...I took hell lot of time figuring out why it is coming up before finally realizing I changed my JDK version.
It might be because JBOSS 7 doesn't work with 1.8 of which I have limited knowledge but yes this might prove useful for some cases.