Binary data over serial terminal - linux

My only way of communication with my embedded device is a serial port. By default, embedded Linux uses this port as a terminal. How do I disable this terminal and use the serial link to transfer binary data? I heard of commands like rx and tx but i cannot find them.
I think I can just read() from and write() stuff to /dev/tty but I want to make sure no error messages or whatever mess with my data stream.

You can use an application like xmodem to transfer file over any terminal. Is the serial port you speak off a terminal, or is it also the kernel console.
If you're kernel is not noisy, then you can use your current connection to make xmodem like transfer. On the host side, you can use kermit, which is nice AND scriptable.
If you want to make your serial port raw, and you have file descriptor ttyfd opened,
here is one way to do it :
struct termios tty, orig_tty;
...
if(tcgetattr(ttyfd, &tty) < 0)
{
// error checking
}
// backup tty, make it raw and apply changes
orig_tty = tty;
cfmakeraw(&tty);
if(tcsetattr(ttyfd, TCSAFLUSH, &tty) < 0)
{
// error checking
}
...
//end of program or error path :
tcsetattr(ttyfd, TCSAFLUSH, &orig_tty)
Don't forget to restore the setting at the end of your program if you still want a good behaved terminal.

Can't you just set the terminal to raw?
Have a look at this tutorial.

To disable the Linux console you have to change the Linux command line create by the bootloader from :
console=/dev/ttyS?
to :
console=null

You can run on the terminal a command that will transfer the data through an application-level protocol. The rx and tx commands you refer to implement the XMODEM file transfer protocol. It could be a solution, if the binary data you want to transfer consists of files, the throughput demands are low, and you don't mind the administrative overhead of running the commands. Alternatively, you may want to multiplex the serial port for handling both data transfer and the terminal. Disable the serial terminal driver command (getty), and run the PPP protocoll over the serial line to establish an IP connection to your device. You can then login to the device through ssh or telnet and transfer your data through an IP socket.

As the other notes have implied, there are several things to check, collected here:
Make sure the linux kernel isn't using the serial port. Make sure there is either no console= option on the bootload command in your grub file. It usually isn't there by default.
Make sure there is no getty running on the serial port. Look in /etc/inittab for an entry for /dev/ttyS0 (serial port A) and comment it out if it is there.
Make sure the /dev/ttyS0 is readable and writable by your process. You might create a specific user under which the control program is run, and which owns the /dev/ttyS0, then chmod 700 /dev/ttyS0. This will help make sure some other user/program doesn't also try using the serial port.
Use open() on ttyS0 to get an fd, then use the tcsetattr family of routines to set the line speed and discipline.
Terminal programs probably won't be useful to you unless you can run the same program on the embedded device to manage the other end of the connection.

Yes, all of your serial ports are in /dev/ttyxx. Note that /dev/tty is a shortcut that stands for your current terminal, not a specific serial port. Often, these are owned by root, and require you to either have privileges or be in the adm group to access the device directly from your application.
You may want to chown the device so you can access it. I'm not sure what the consequence of changing device ownership are; IIRC, it's easy to do and works nicely. The alternative is to use setuid to elevate your program to a privileged state.
There's a program named getty that lets users login from a serial port. Your inittab will start getty on serial ports so people can login.
You want to disable getty. In some cases, there's a port manager that helps do this.
In some cases, you can change your inittab to use mgetty, which is a smarter and easier to control version of getty.

Related

Retrieve variable (or state file) from remote host using telnet?

I have wifi module as client that connected to my router (linux based firmware, Openwrt).
And sometime, i need to retrieve GPIO state from wifi module. It can be done simply by connect in to usb serial and type
print (gpio.read(1))
It will return value 1 or 0. Active or not.
The thing is, can i save the value of gpio to my openwrt through telnet?
Using ash or bash maybe?
I've succesfully connected to wifi module using telnet and execute command via telnet.
Many thanks :)
Note :
-my module wifi is esp8266-07,
-Router HW Echolife HG553 (openwrt,Debian based linux)
Linux doesn't provide us with any "standardtized" solution.
You would need to implement some kind of interface, e.g. service that will read data and make nicely structured JSON file, which is shared over HTTP, so you can read and parse it.

IP tunnel over Linux serial default shell

This is a more constrained version of this question:
I have an embedded ARM device running a custom image with a Linux 3.10.0 kernel.
The only physical interface (no, USB, no Ethernet) is the default Linux shell which is connected one of the serial interfaces.
My question is: Is there any built-in or external tool that opens an IP tunnel over this connection?
I see some general issues:
The device is already use by Linux, so it must use stdin/out to communicate instead of accessing the device directly.
After starting the tunneling application, the application must wait for a tunnel client to connect because I need to close the serial connection on my computer and then start the tunnel client.
There should be a way to close the connection and go back to the normal shell
The actual requirement is, that I can access a REST interface that is running on the embedded device from a computer connected to the embedded device via serial cable.
This already works on devices with a physical Ethernet or Ethernet-over-USB but this device does not offer that.
[UPDATE]
As explained, socat is currently not available on our embedded device so as a first attempt, I used the following:
A Linux (Ubuntu) laptop with a physical serial interface
A Windows Laptop with a physical serial interface and cygwin+socat installed
Both connected via Null-modem cable
Note: I'm using a Windows laptop on one side because we will have the socat client running on Linux (unfortunately).
Direct STDIO Connection
Server
socat stdio file:/dev/ttyS0,b115200
Client
socat file:/dev/ttyS4,b115200 stdio
In cygwin, ttyS0 is COM1, ttyS4 in this case is COM5.
Using these, socat works like a little chat program. Why I type on one side is output on the other and vice-versa.
TCP Connection
The next step is to use a TCP connection.
Server
socat /dev/ttyS0,b115200,crtscts=1,raw,echo=0 tcp-connect:localhost:80
Client
socat -T2 file:/dev/ttyS4,b115200,crtscts=1,raw,echo=0 tcp-l:7777,reuseaddr
I specified the baud rate (115200), used raw transmission, no echo (The HTTP request would otherwise be sent back to the requester) using hardware flow control. Pus I had to use a timeout -T2 wich terminates the connection after 2s. Otherwise, curl does not terminate either and waits for more data.
When I use curl on the windows computer, it successfully transmits the request over serial connection and returns the complete HTTP response of the HTTP server on the Linux computer:
curl localhost:7777/index.html
However, it works only once. After the request is completed, both socatclient and server terminates.
Moreover, when I use a browser (Chorme), it uses g-zip encoding which most probably sends binary characters. And one of these characters will be a EOF character which again terminates socat before completing the request/response.
Then I tried to add fork to the server:
socat /dev/ttyS0,b115200,crtscts=1,raw,echo=0 tcp-connect:localhost:80,fork
This keeps the server alive, but curl returns a 400 Bad Request. So it seems as if the socat server initiated a request for each line or chunk since it does not understand HTTP.
IP Connection
Then I thought about going a layer below and using a TUN connection. However, this is not implemented on the Windows version of socat.
HTTP connection
Correct me if I'm wrong, but as far as I understand, socatdoes not provide a connection type that actually understands HTTP and is able to serialize it properly over a serial connection.
So, I couldn't find any stable way to start both client and server and run multiple HTTP requests over the serial connection.
On a normal linux, you could use socat.
This program allows you to connect several stream types (file, socket, tcp, udp, ...). In your case it would be tcp to file or more precisely a tcp socket at port xx to /dev/ttyUSB1. You should launch socat on both sides to build a tunnel.
Edit 1:
Sorry I got also disappointed by socat. I can't find a solution that keeps my TCP listener active for multiple successive connections, but handles only one connection at a time.
My solution is a simple C# program that uses 4 threads:
1. wait for input on stdin e.g. exit command
2. the TCP listener
3. the TCP worker thread for a active connection
4. if TCP is open, it opens another thread for COM
Thread 3 reads from TCP and writes to COM and Tread 4 reads from COM and writes to TCP. If thread gets a TCP close event, it stops thread 4, which closes COMx, and exits it self. Now thread 2 can accept a new connection. If thread 1 reads exit on stdin, it passes a message to all threads to stop and shutdown.
Maybe you can implement such a short program in C with pthreads on your embedded system, which has no socat.
The EOF problem:
I tried to google for a program that escapes a special character or reencodes a data stream from ASCII to ANSI or base64 or whatever.... If you can find such a program or write it also in C you can pipe it in between
Server <=> reencode <=> socat <--serial--> socat <=> reencode <=> client
We've now solved the problem halfway using pppd. As it turns out, even Windows supports ppp. In contrast to socat, pppd actually uses a protocol that will have error detection included and it automatically creates network devices on the Linux and Windows system.
The only problem is, that pppd requires to have access to the serial device. There is no direct mode like the ppp tool provides.
We are now disabling the shell on demand, rebooting into IP-over-serial mode. When we are done, we reboot the system which automatically switch back to getty using the serial line.
The is not the prettiest solution but right now, it seems to work.

How to use the in-qemu API for virtio serial device

I want to exchange "complex" data (hierarchical structs) between the guest and the host in a KVM+Qemu setup.
My idea was to use the virtio serial driver to do so. A guest application would use the normal I/O functions such as open(2), close(2), read(2) and write(2) to send the "buffer" (=the struct I want to send) to the virtio serial back-end. The back-end driver would run inside Qemu and receive the pointer to the struct through the in-qemu host API described here: http://www.linux-kvm.org/page/Virtio-serial_API
My question now may be relatively trivial, but I was looking around all over Google and couldn't find anything: How do I "hook into qemu" such that I can use the virtio serial host API?
I understand that I have to provide an init function like void my_init(void) { virtio_serial_port_qdev_register(&myinfo); } and register it with qemu using device_init(&my_init). However, I do not understand how I link my module with qemu? Do I have to add my source files to the qemu code base and makefiles and recompile qemu? Or can I compile it separately and use some magic qemu command line option to load it? Or something completely different?
(Note: I know that I could optionally serialize my data, send it to a socket on the host, and de-serialize it there, but I wanted to avoid the serialization overhead. If there is a way to use a shared memory region instead of a socket with an out-of-the-box virtIO serial device, this could be an option, too).
Thank you all for your help!
Christoph
you can start vm using
qemu-system-x86_64 -m 1024 -name mac -hda ~/Documents/ubuntu -device virtio-serial -chardev socket,path=/tmp/foo,server,nowait,id=foo -device virtconsole,name=jobsfoo,chardev=foo,name=org.fedoraproject.console.foo
and transfer data by using
socat /tmp/foo (host)
socat /tmp/hvc0 (guest)
or you can use socket program on host and simple file i/o on guest
on the host side a serial port can be attached to a pipe or socket using the "-chardev" option. If you specify for example "-chardev socket,path=/tmp/foo.sock,server,nowait,id=foo", you can connect a virtio-serial port to that socket with "-device virtserialport,chardev=foo". You do not need to modify QEMU (or I don't understand what you want to do).

Notify me when a socket binds, like inotify does for files

I am interested in finding out when things SSH into my boxen to create a reverse tunnel. Currently I'm using a big hack - just lsof with a few lines of script. So my goal is to see when a socket calls bind() and, ideally, get the port it binds to (it's listening locally since it's a reverse tunnel) and the remote host that I would be connecting to. My lsof hack is basically fine, except I don't get instant notifications and it's rather... hacky :)
This is easy for files; once a file does just about anything, inotify can tell me in Linux. Of course, other OSs have a similar capability.
I'm considering simply tailing the SSHD logs and parsing the output, but my little "tunnel monitor" daemon needs to be able to figure out the state of the tunnels at any point in time, even if it hasn't been running the whole time SSHD has.
I have a pretty evil hack I've been considering as well. It's a script that invokes GDB on /usr/sbin/sshd, then sets a breakpoint on bind. Then it runs it with the options -d -p <listening port> -- Running a separate SSHD for these tunnels is fine. Then it waits for that breakpoint to get hit, and uses GDB's input to get the remote hosts's IP address and the local IP on which SSH is now listening. Again, that's text parsing and opens some other issues.
Is there a "good" way to do this?
I would use SystemTap for a problem like this. You can use it to probe the kernel to see when a bind is done by any process on the system. http://sourceware.org/systemtap/

How create a virtual io device in Linux that proxies data to real device?

I have an interesting problem. I am working on an embedded box with multiple instances of Linux running each on an ARM processor. They are connected over internal 1GBps network. I have a serial port device node attached to processor A (Lets say Linux-A running on it). I have a program running on processor B (Lets say on Linux-B) access the serial port device as if it is attached to Linux-B locally.
My program invokes term i/o type api calls on device node to control tty echo, character mode input. What I am wondering is if there is a way to create a virtual serial device that is available on Linux-B somehow talking to real serial device on Linux-A over internal network.
I am thinking something along the lines of:
Linux-B has /dev/ttyvirtual. Anything that gets written to it gets transported over network socket to Linux-A serialserver. The serial server exrcises the api calls on real device lets say /dev/ttys0.
Any data waiting on ttys0 gets transported back to /dev/ttyvirtual.
What are all the things involved to get this done fast?
Thanks
Videoguy
Update:
I found a discussion at
http://fixunix.com/bsd/261068-network-socket-serial-port-question.html with great pointers.
Another useful link is http://blog.philippklaus.de/2011/08/make-rs232-serial-devices-accessible-via-ethernet/
Take a look at openpty(3). This lets you create a pseudo-TTY (like /dev/pts/0, the sort that ssh connections use), which will respond as a normal TTY would, but give you direct programmatic control over the connections.
This way you can host a serial device (eg. /dev/pts/5) that you forward data between a network connection, and then other apps can perform serial operations on it without knowing about the underlying network bridge.
I ended up using socat
Examples can be found here: socat examples
You socat back to back on both the machines. One listens on a tcp port and forwards data to local virtual port or pty. The socat on other box uses real device as input and forwards any data to tcp port.

Resources