I program distributed program in C++ that uses TCP and run it on linux cent os 7 with kernel 3.1.0
The program is built for high performance with high CPU, disk and network usage.
The program might run over a few days like 4 days. I am worried about the case where TCP connection is lost during the computation for any reason except for the case that one of machines died.
Can this happen? (The tcp connection is lost while the machines are all alive and no one invoked close on the socket?)
If possible, what can the programmer like me do for it?
Can I detect the lost connection and try to reconnect it?
Thanks,
Ideally, connection management is part of the protocol. This way the management is documented and client and server know what is expected.
Some strategies:
use UDP: no connection. Handle request/reply and possible failures. Timeout handling of reply may be needed.
short TCP connections: only connect when needed and disconnect after "transaction" (e.g. http)
long TCP connection with keep-alive checks and connection retries: check for connection failures and have client reconnect and servers wait for reconnection.
This is a more constrained version of this question:
I have an embedded ARM device running a custom image with a Linux 3.10.0 kernel.
The only physical interface (no, USB, no Ethernet) is the default Linux shell which is connected one of the serial interfaces.
My question is: Is there any built-in or external tool that opens an IP tunnel over this connection?
I see some general issues:
The device is already use by Linux, so it must use stdin/out to communicate instead of accessing the device directly.
After starting the tunneling application, the application must wait for a tunnel client to connect because I need to close the serial connection on my computer and then start the tunnel client.
There should be a way to close the connection and go back to the normal shell
The actual requirement is, that I can access a REST interface that is running on the embedded device from a computer connected to the embedded device via serial cable.
This already works on devices with a physical Ethernet or Ethernet-over-USB but this device does not offer that.
[UPDATE]
As explained, socat is currently not available on our embedded device so as a first attempt, I used the following:
A Linux (Ubuntu) laptop with a physical serial interface
A Windows Laptop with a physical serial interface and cygwin+socat installed
Both connected via Null-modem cable
Note: I'm using a Windows laptop on one side because we will have the socat client running on Linux (unfortunately).
Direct STDIO Connection
Server
socat stdio file:/dev/ttyS0,b115200
Client
socat file:/dev/ttyS4,b115200 stdio
In cygwin, ttyS0 is COM1, ttyS4 in this case is COM5.
Using these, socat works like a little chat program. Why I type on one side is output on the other and vice-versa.
TCP Connection
The next step is to use a TCP connection.
Server
socat /dev/ttyS0,b115200,crtscts=1,raw,echo=0 tcp-connect:localhost:80
Client
socat -T2 file:/dev/ttyS4,b115200,crtscts=1,raw,echo=0 tcp-l:7777,reuseaddr
I specified the baud rate (115200), used raw transmission, no echo (The HTTP request would otherwise be sent back to the requester) using hardware flow control. Pus I had to use a timeout -T2 wich terminates the connection after 2s. Otherwise, curl does not terminate either and waits for more data.
When I use curl on the windows computer, it successfully transmits the request over serial connection and returns the complete HTTP response of the HTTP server on the Linux computer:
curl localhost:7777/index.html
However, it works only once. After the request is completed, both socatclient and server terminates.
Moreover, when I use a browser (Chorme), it uses g-zip encoding which most probably sends binary characters. And one of these characters will be a EOF character which again terminates socat before completing the request/response.
Then I tried to add fork to the server:
socat /dev/ttyS0,b115200,crtscts=1,raw,echo=0 tcp-connect:localhost:80,fork
This keeps the server alive, but curl returns a 400 Bad Request. So it seems as if the socat server initiated a request for each line or chunk since it does not understand HTTP.
IP Connection
Then I thought about going a layer below and using a TUN connection. However, this is not implemented on the Windows version of socat.
HTTP connection
Correct me if I'm wrong, but as far as I understand, socatdoes not provide a connection type that actually understands HTTP and is able to serialize it properly over a serial connection.
So, I couldn't find any stable way to start both client and server and run multiple HTTP requests over the serial connection.
On a normal linux, you could use socat.
This program allows you to connect several stream types (file, socket, tcp, udp, ...). In your case it would be tcp to file or more precisely a tcp socket at port xx to /dev/ttyUSB1. You should launch socat on both sides to build a tunnel.
Edit 1:
Sorry I got also disappointed by socat. I can't find a solution that keeps my TCP listener active for multiple successive connections, but handles only one connection at a time.
My solution is a simple C# program that uses 4 threads:
1. wait for input on stdin e.g. exit command
2. the TCP listener
3. the TCP worker thread for a active connection
4. if TCP is open, it opens another thread for COM
Thread 3 reads from TCP and writes to COM and Tread 4 reads from COM and writes to TCP. If thread gets a TCP close event, it stops thread 4, which closes COMx, and exits it self. Now thread 2 can accept a new connection. If thread 1 reads exit on stdin, it passes a message to all threads to stop and shutdown.
Maybe you can implement such a short program in C with pthreads on your embedded system, which has no socat.
The EOF problem:
I tried to google for a program that escapes a special character or reencodes a data stream from ASCII to ANSI or base64 or whatever.... If you can find such a program or write it also in C you can pipe it in between
Server <=> reencode <=> socat <--serial--> socat <=> reencode <=> client
We've now solved the problem halfway using pppd. As it turns out, even Windows supports ppp. In contrast to socat, pppd actually uses a protocol that will have error detection included and it automatically creates network devices on the Linux and Windows system.
The only problem is, that pppd requires to have access to the serial device. There is no direct mode like the ppp tool provides.
We are now disabling the shell on demand, rebooting into IP-over-serial mode. When we are done, we reboot the system which automatically switch back to getty using the serial line.
The is not the prettiest solution but right now, it seems to work.
I am working on a C program on Linux (kernel 2.6.18). I need to send/inject IP packets (e.g., over a socket) in my Linux systems, but make the same Linux "think" that these packets are incoming from another host. I creat a datalink socket and use faked source mac/ip for the packets sent over this socket. The destination mac/ip are set to the ones in my local Linux. However, whether I send these packets in a user-space program or in a kernel module, my local Linux just doesn't think these packets are coming from outside. For example, if I create a datalink socket to send an ICMP request destined to my local Linux, I expect my local Linux to think this ICMP request coming from outside, and would respond with an ICMP reply, but my local Linux does not do so. (However, with the same program I can send a faked ICMP request to another host, and that host does respond an ICMP reply.)
I did some research on this topic online, and it seems all related solution suggest using TAP. But as this VirtualBox article says:
... TAP is no longer necessary on Linux with bridged networking, ...
I am very interested to know how this is possible. Thanks.
I have an interesting problem. I am working on an embedded box with multiple instances of Linux running each on an ARM processor. They are connected over internal 1GBps network. I have a serial port device node attached to processor A (Lets say Linux-A running on it). I have a program running on processor B (Lets say on Linux-B) access the serial port device as if it is attached to Linux-B locally.
My program invokes term i/o type api calls on device node to control tty echo, character mode input. What I am wondering is if there is a way to create a virtual serial device that is available on Linux-B somehow talking to real serial device on Linux-A over internal network.
I am thinking something along the lines of:
Linux-B has /dev/ttyvirtual. Anything that gets written to it gets transported over network socket to Linux-A serialserver. The serial server exrcises the api calls on real device lets say /dev/ttys0.
Any data waiting on ttys0 gets transported back to /dev/ttyvirtual.
What are all the things involved to get this done fast?
Thanks
Videoguy
Update:
I found a discussion at
http://fixunix.com/bsd/261068-network-socket-serial-port-question.html with great pointers.
Another useful link is http://blog.philippklaus.de/2011/08/make-rs232-serial-devices-accessible-via-ethernet/
Take a look at openpty(3). This lets you create a pseudo-TTY (like /dev/pts/0, the sort that ssh connections use), which will respond as a normal TTY would, but give you direct programmatic control over the connections.
This way you can host a serial device (eg. /dev/pts/5) that you forward data between a network connection, and then other apps can perform serial operations on it without knowing about the underlying network bridge.
I ended up using socat
Examples can be found here: socat examples
You socat back to back on both the machines. One listens on a tcp port and forwards data to local virtual port or pty. The socat on other box uses real device as input and forwards any data to tcp port.
Is it possible in Node.JS to "drop" a connection in such a way that
The client never receives a response (200, 404 or otherwise)
The client is never notified that the connection is terminated (never receives connection reset or end of stream)
The server's resources are released (the server should not attempt to maintain the connection in any way)
I am specifically asking about Node.JS HTTP Servers (which are really just complex TCP servers) on Solaris., but if there are cases on other OSes (Windows, Linux) or programming languages (C/C++, Java) that permit this, I am interested.
Why do I want this?
To annoy or slow down (possibly single-threaded) robots such as phpMyAdmin Probe.
I know this is not really something that matters, but these types of questions can better help me learn the boundaries of my programs.
I am aware that the client host is likely to re-transmit the packets of the connection since I am never sending reset.
These are not possible in a generic TCP stack (vs. your own custom TCP stack). The reasons are:
Closing a socket sends a RST
Even if you avoid sending a RST, the client continues to think the connection is open while the server has closed the connection. If the client sends any packet on this connection, the server is going to send a RST.
You may want to explore firewalling these robots and block / rate limit their IP addresses with something like iptables (linux) or the equivalent on solaris.
closing a connection should NOT send an RST. There is a 3 way tear down process.