IP tunnel over Linux serial default shell - linux

Question

This is a more constrained version of this question:
I have an embedded ARM device running a custom image with a Linux 3.10.0 kernel.
The only physical interface (no, USB, no Ethernet) is the default Linux shell which is connected one of the serial interfaces.
My question is: Is there any built-in or external tool that opens an IP tunnel over this connection?
I see some general issues:
The device is already use by Linux, so it must use stdin/out to communicate instead of accessing the device directly.
After starting the tunneling application, the application must wait for a tunnel client to connect because I need to close the serial connection on my computer and then start the tunnel client.
There should be a way to close the connection and go back to the normal shell
The actual requirement is, that I can access a REST interface that is running on the embedded device from a computer connected to the embedded device via serial cable.
This already works on devices with a physical Ethernet or Ethernet-over-USB but this device does not offer that.
[UPDATE]
As explained, socat is currently not available on our embedded device so as a first attempt, I used the following:
A Linux (Ubuntu) laptop with a physical serial interface
A Windows Laptop with a physical serial interface and cygwin+socat installed
Both connected via Null-modem cable
Note: I'm using a Windows laptop on one side because we will have the socat client running on Linux (unfortunately).
Direct STDIO Connection
Server
socat stdio file:/dev/ttyS0,b115200
Client
socat file:/dev/ttyS4,b115200 stdio
In cygwin, ttyS0 is COM1, ttyS4 in this case is COM5.
Using these, socat works like a little chat program. Why I type on one side is output on the other and vice-versa.
TCP Connection
The next step is to use a TCP connection.
Server
socat /dev/ttyS0,b115200,crtscts=1,raw,echo=0 tcp-connect:localhost:80
Client
socat -T2 file:/dev/ttyS4,b115200,crtscts=1,raw,echo=0 tcp-l:7777,reuseaddr
I specified the baud rate (115200), used raw transmission, no echo (The HTTP request would otherwise be sent back to the requester) using hardware flow control. Pus I had to use a timeout -T2 wich terminates the connection after 2s. Otherwise, curl does not terminate either and waits for more data.
When I use curl on the windows computer, it successfully transmits the request over serial connection and returns the complete HTTP response of the HTTP server on the Linux computer:
curl localhost:7777/index.html
However, it works only once. After the request is completed, both socatclient and server terminates.
Moreover, when I use a browser (Chorme), it uses g-zip encoding which most probably sends binary characters. And one of these characters will be a EOF character which again terminates socat before completing the request/response.
Then I tried to add fork to the server:
socat /dev/ttyS0,b115200,crtscts=1,raw,echo=0 tcp-connect:localhost:80,fork
This keeps the server alive, but curl returns a 400 Bad Request. So it seems as if the socat server initiated a request for each line or chunk since it does not understand HTTP.
IP Connection
Then I thought about going a layer below and using a TUN connection. However, this is not implemented on the Windows version of socat.
HTTP connection
Correct me if I'm wrong, but as far as I understand, socatdoes not provide a connection type that actually understands HTTP and is able to serialize it properly over a serial connection.
So, I couldn't find any stable way to start both client and server and run multiple HTTP requests over the serial connection.

Answer 1

On a normal linux, you could use socat.
This program allows you to connect several stream types (file, socket, tcp, udp, ...). In your case it would be tcp to file or more precisely a tcp socket at port xx to /dev/ttyUSB1. You should launch socat on both sides to build a tunnel.
Edit 1:
Sorry I got also disappointed by socat. I can't find a solution that keeps my TCP listener active for multiple successive connections, but handles only one connection at a time.
My solution is a simple C# program that uses 4 threads:
1. wait for input on stdin e.g. exit command
2. the TCP listener
3. the TCP worker thread for a active connection
4. if TCP is open, it opens another thread for COM
Thread 3 reads from TCP and writes to COM and Tread 4 reads from COM and writes to TCP. If thread gets a TCP close event, it stops thread 4, which closes COMx, and exits it self. Now thread 2 can accept a new connection. If thread 1 reads exit on stdin, it passes a message to all threads to stop and shutdown.
Maybe you can implement such a short program in C with pthreads on your embedded system, which has no socat.
The EOF problem:
I tried to google for a program that escapes a special character or reencodes a data stream from ASCII to ANSI or base64 or whatever.... If you can find such a program or write it also in C you can pipe it in between
Server <=> reencode <=> socat <--serial--> socat <=> reencode <=> client

Answer 2

We've now solved the problem halfway using pppd. As it turns out, even Windows supports ppp. In contrast to socat, pppd actually uses a protocol that will have error detection included and it automatically creates network devices on the Linux and Windows system.
The only problem is, that pppd requires to have access to the serial device. There is no direct mode like the ppp tool provides.
We are now disabling the shell on demand, rebooting into IP-over-serial mode. When we are done, we reboot the system which automatically switch back to getty using the serial line.
The is not the prettiest solution but right now, it seems to work.