Create a virtual serial port connection over TCP - linux

I am developing an application that should be able to write to a virtual serial port and receive data through the same port from remote clients over network.
The application runs on a linux server. I am new in using serial ports and I have some questions on this topic.
Clients
The client can establish a TCP connection to a server. When we setup a client, we have to provide the IP address of the server, a tcp port (usually 8080) and a virtual com port.
The client then will automatically try to connect to the server.
Server
The server has a virtual com port, the same we set in the client config (e.g. COM1). When an application on the server writes data to this port, the data should be send to all clients connected via tcp. The response from the clients is send over TCP back to the server which can read it over the virtual serial port.
Question
On windows I used a virtual serial port connector http://www.eterlogic.com/Products.VSPE.html which did most of the work. However I want to solve this problem on linux machines.
My question is, how can I create a TCP server that has a virtual serial port attached and can send/receive data through this port over TCP to listening clients?

Try socat. Possible scenario:
socat pty,link=/dev/virtualcom0,raw tcp:192.168.254.254:8080&
socat creates TCP connection to 192.168.254.254:8080, so that everything, that will be written to /dev/virtualcom0 will be forwarded to 192.168.254.254:8080 and vice versa.
Another approach would be to use RFC2217 via ser2net on Linux sever side and RFC2217 driver on Windows side (for example http://www.hw-group.com/products/hw_vsp/index_en.html single port version). You can also try to get http://pyserial.sourceforge.net/ to work with ser2net.

you have socat and ser2net and other programs but my experience is very bad... not working properly. I've done this small python program, can be useful. Update port, baudrate... then use any tcp client. Remove first line if don't want to use is as auto executable script
#!/usr/bin/python
import socket
import sys
import serial
#open serial port
ser = serial.Serial('/dev/ttyAMA0', 115200, timeout=0)
#create socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
#bond to the port. Don't use localhost to accept external connections
server_address = ('', 2105)
print('starting up on {} port {}'.format(*server_address))
sock.bind(server_address)
#listen
sock.listen(1)
#loop
while True:
#waits for a new connection
print('waiting for a connection')
connection, client_address = sock.accept()
try:
print('connection from', client_address)
#continously send from serial port to tcp and viceversa
connection.settimeout(0.1)
while True:
try:
data = connection.recv(16)
if data == '': break
ser.write(data)
except KeyboardInterrupt:
connection.close()
sys.exit()
except Exception as e:
pass
received_data = ser.read(ser.inWaiting())
connection.sendall(received_data)
except Exception as e:
print e
finally:
#clean up connection
connection.close()

The software will help to establish server and client connection over TCP http://www.serial-com-port.com/
I use it for creating virtual serial communications over network, but I have the real RS232 port on the computer. So I just transfer the data over network. If you need to create a virtual COM on the server too, use the Virtual Serial Port Driver.

Related

Why is "connection refused" error occuring

This is client.py class and This class is on mobile.
import socket
c = socket.socket()
c.connect(('127.0.0.1', 9998))
name = input("Enter your name : ")
c.send(bytes(name, 'utf-8'))
data = c.recv(1024).decode()
print(data)
This is server.py class and This class is on mac book pro.
import socket
s = socket.socket()
s.bind(('localhost', 9998))
s.listen(2)
print('server is waiting for connections..')
while True:
c, addr = s.accept()
name = c.recv(1024).decode()
print('connected with', addr, name)
st = "Hello, " + name
c.send(st)
c.close()
Why am I getting "connection refused" error?
Your client and server are running on different devices. Your client is trying to connect to the local loopback IP 127.0.0.1, which will work only if the server is running on the same device as the client. That is why you are getting "connection refused" - there is no server listening locally at 127.0.0.1:9998.
If the client and server are connected to the same network (WiFi, etc), the client needs to connect to the server's actual LAN IP on that network.
If the client and server are connected to different networks (ie, they reach each other over the Internet), the client needs to connect to the public IP of the network that the server is connected to, and that network router will need to have port forwarding configured to route incoming connections to the server device.
Also, note that making the server listen on localhost does not guarantee it will be able to accept clients from other devices, depending on how localhost is implemented. It might resolve to 127.0.0.1 only. The server should instead listen on the wildcard IP 0.0.0.0 so it listens on all available network interfaces that are installed on that device. Or, it can alternatively listen on just the specific LAN IP that will actually be receiving the client connections.
To echo what Remy just correctly said, "connection refused" is really a bit misleading. It basically means that there was no one listening at the address-and-port that the host tried to reach. It does not mean that "there was someone there, but he [actively ...] 'refused to' talk with you." "Connection failed" might have been a better term, but, here we are.

Why is my Socket Client not allowing the outgoing connection in python 3?

I have created a very simple UDP Socket client and server in python3.
I'm trying to send the simple message 'hello' to the server and I am getting the error:
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it
I'm not sure that it's actually the target machine's fault as the target machine is an AWS EC2 Ubuntu instance with 'All Traffic Allowed' configured with the security group.
I'm convinced it has to be some sort of outbound policy on my desktop computer...
Any help is appreciated.
Here is my code:
client.py
import socket
client_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client_sock.connect(('servers public ipv4 ip', 8089))
client_sock.send('hello'.encode())
server.py
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(('localhost', 8089))
s.listen(5) #5 connections
while True:
conn, addr = s.accept()
buffer = conn.recv(64)
if len(buffer) > 0:
print(buffer.decode())
break
First, despite your repeated claims (in body and tags) that you are using UDP you are actually using TCP as can be seen from your use of SOCK_STREAM and accept.
s.bind(('localhost', 8089))
You explicitly bind the server to localhost. But localhost is only the loopback interface of the computer (127.0.0.1) which is not reachable from outside. If you want to accept connections from other systems you need to either bind to the IP address of the specific network interface or broadly to 0.0.0.0 - or simply use s.bind('',8089).

Basic UDP networking doesn't receive

I'm trying to learn UDP networking so I tried the simplest code to begin with. It's a python code client-server that works perfectly when I send data to localhost or to the LAN IP from the same computer, but it doesn't work when I try to send from my computer using the public IP, and also doesn't work from another computer using private network IP or public IP.
I did the port forwarding, created the input rules for windows firewall, turned off the router internal firewall, and it still doesn't work.
When I scan my port from canyouseeme.org or using netcat it says connection refused and port closed.
Maybe somebody can guess what is happening here or what may I do to succeed?
I write the code below in case it's needed.
Server:
import socket
UDP_IP = "0.0.0.0"
UDP_PORT = XXXX
sock = socket.socket(socket.AF_INET, # Internet
socket.SOCK_DGRAM) # UDP
sock.bind((UDP_IP, UDP_PORT))
while True:
data, addr = sock.recvfrom(1024) # buffer size is 1024 bytes
message = data.decode()
print("received message:", message)
Client:
import socket
UDP_IP = "192.168.1.133"
UDP_PORT = XXXX
MESSAGE = "Hello, World!"
print("UDP target IP:", UDP_IP)
print("UDP target port:", UDP_PORT)
print("message:", MESSAGE)
sock = socket.socket(socket.AF_INET, # Internet
socket.SOCK_DGRAM) # UDP
sock.sendto(MESSAGE.encode(), (UDP_IP, UDP_PORT))
Thanks in advance.
After some investigation I have found a partial solution. I'm posting an answer because it may help some people with similar problems, but it's not the best solution yet, so the issue is still there.
I found that there's no problem in communicating within a LAN if I completely disable the firewall in both computers. I don't know why the input rules or exceptions don't work.
For networking beyond the LAN I have tried successfully to use a virtual LAN (i.e. hamachi) and there's no necessity of port forwarding nor disabling the firewall. The bad thing about hamachi is that I don't control the protocol it uses, and I don't control the way of ensuring reliability.
For the applications I'm dealing with right now I probably have enough with this solution, but it would be interesting to know more.

Receiving UDP broadcast packets on Linux

We have existing software that periodically broadcasts UDP packets to a specific port (7125) on the local subnet (x.x.x.255). We have monitoring software running on HP-UX (11.11) that is able to receive these packets no problem. However, after porting the monitoring software to Linux (RHEL 6.1) we have found that it does not receive the broadcast packets. tcpdump shows the packets arriving at the Linux host, but the kernel does not send them through to our software.
I've been using a couple of python 2.x scripts that mimic the socket API calls the monitoring software uses to test different scenarios. The Linux kernel passes the packets to the receiver software if the sender uses unicast (10.1.0.5), but not broadcast (10.1.0.255). I've been searching the web for several days and have not found anyone with the same problem. Any ideas?
receiver.py
from __future__ import print_function
import socket
localHost = ''
localPort = 7125
remoteHost = '10.1.0.5'
remotePort = 19100
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)
s.bind((localHost, localPort))
s.connect((remoteHost, remotePort))
print('Listening on {0}:{1} for traffic from {2}:{3}'.format(localHost, localPort, remoteHost, remotePort))
data = s.recv(1024)
print('Received: {0}'.format(data))
s.close()
sender.py
from __future__ import print_function
import socket
import time
localHost = ''
localPort = 19100
remoteHost = '10.1.0.255'
remotePort = 7125
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)
s.bind((localHost, localPort))
s.connect((remoteHost, remotePort))
data = 'sending this from {0}:{1} to {2}:{3}'.format(localHost, localPort, remoteHost, remotePort)
print(data)
print('2')
time.sleep(1)
print('1')
time.sleep(1)
s.send(data)
print('sent at {0}'.format(time.ctime()))
s.close()
Well, I suggested this answer in a comment, and it proved correct in practice. I would like to investigate surrounding nuances further with my own code, but this is the canonical case-closer.
In addition to setting the SO_BROADCAST socket option on both sides (as you are already correctly doing), you must also bind your receiver to the broadcast address (e.g., INADDR_BROADCAST, which is 255.255.255.255 and essentially serves the same role as INADDR_ANY for unicast).
Apparently, in the HP-UX configuration of the original poster, a UDP socket bound to a unicast address (or INADDR_ANY, specifically) but with the SO_BROADCAST socket option set will still receive all UDP datagrams addressed to the local broadcast address as well as unicast traffic directed at the host.
Under Linux, this is not the case. Binding a UDP socket, even when SO_BROADCAST-enabled, to INADDR_ANY is insufficient to receive both unicast and broadcast datagrams on the bound port. One can use a separate INADDR_BROADCAST-bound SO_BROADCAST socket for the broadcast traffic.

Socket getting created with same IP and port on local host

I am seeing weird behavior on Linux where I am seeing that remote end and local end are both showing same IP and port combination. Following is the netstat output
netstat -anp | grep 6102
tcp 0 0 139.185.44.123:61020 0.0.0.0:* LISTEN 3361/a.out
tcp 0 0 139.185.44.123:61021 139.185.44.123:61021 ESTABLISHED 3361/a.out
Can anyone tell me if this is even possible ? If yes, then what could be the scenario ?
A connection is identified by a 4-tuple ((source ip, source port), (target ip, target port)), and the source and target ports could conceivably be the same without any issues. This connection could even be established by one process, which would lead to the output you're seeing.
However, I just got bitten by a nasty bug, where a client socket would try to connect to a server socket with a port number in the ephemeral port range on the same machine. The connection operation would be retried operation until it succeeded.
The retry feature was the issue: if the server application wasn't running AND the source port that got picked at random was the same as the target port (which is possible because the target port was in the ephemeral range), the client socket would CONNECT TO ITSELF (which was wreaking havoc on the internal logic of the client application, you can imagine).
Since the client was performing retries as quickly as possible, the 1 in 30.000 odds that this can happen were hit quickly enough.
The following Python script reproduces it:
import socket
host = 'localhost'
port = 35911
ctr = 0
while True:
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.settimeout(5)
ctr += 1
s.connect((host, port))
print "Connected to self after", ctr, "tries"
break
except socket.error, e:
print e
# Retry
Here is an imaginable scenario. The caller of connect could call bind before calling connect. The caller of connect could cooperate with the caller of listen, and intentionally bind to a port number 1 higher than the listening port number.
If the kernel proceeds to reuse the socket number for the listener, I'd call that a kernel bug.
When multi-threaded server software accepts connection, it usually creates another socket, which communicates with newly connected client in separate thread, while original server socket is still listening for new clients in original thread. In such cases ports of both sockets are equal. So, there's no any problem.
It's a slightly odd case, called a TCP "active/active open". A socket has been opened, bound to a local port and then connected to itself, by using connect() with its own address as the destination.
Nothing weird about that. It has a 1 in 63k chance of happening. What you won't see is * two* such ESTABLISHED* connections: that's impossible by the rules of TCP.

Resources