I'm debugging communications with a serial device, and I need to see all the data flowing both directions.
It seems like this should be easy on Linux, where the serial port is represented by a file. Is there some way that I can do a sort of "bi-directional tee", where I tell my program to connect to a pipe that copies the data to a file and also shuffles it to/from the actual serial port device?
I think I might even know how to write such a beast, but it seems non-trivial, especially to get all of the ioctls passed through for port configuration, etc.
Has anyone already built such a thing? It seems too useful (for people debugging serial device drivers) not to exist already.
strace is very useful for this. You have a visualisation of all ioctl calls, with the corresponding structure decoded. The following options seems particularly useful in your case:
-e read=set
Perform a full hexadecimal and ASCII dump of all the data read from
file descriptors listed in the
specified set. For example, to see all
input activity on file descriptors 3
and 5 use -e read=3,5. Note that this
is independent from the normal tracing
of the read(2) system call which is
controlled by the option -e
trace=read.
-e write=set
Perform a full hexadecimal and ASCII
dump of all the data written to file
descriptors listed in the specified
set. For example, to see all output
activity on file descriptors 3 and 5
use -e write=3,5. Note that this is
independent from the normal tracing of
the write(2) system call which is
controlled by the option -e
trace=write.
I have found pyserial to be quite usable, so if you're into Python it shouldn't be too hard to write such a thing.
A simple method would be to write an application which opened
the master side of a pty and the tty under test. You would then
pass your tty application the slave side of the pty as the 'tty device'.
You would have to monitor the pty attributes with tcgetattr() on the pty
master and call tcsetattr() on the real tty, if the attributes changed.
The rest would be a simple select() on both fd's copying data bi-directionally and copying it to a log.
I looked at a lot of serial sniffers. All of them are based on the idea of making a virtual serial port and sniff data from that port. However, any baud/parity/flow changes will break connection.
So, I wrote my own sniffer :). Most of the serial ports now are just USB-to-serial converters. My sniffer collects data from USB through debugfs, parse it and output to the console. Also any baudrate changes, flow control, line events, and serial errors are also recorded. The project is in the early stage of development and for now, only FTDI is supported.
http://code.google.com/p/uscmon/
Much like #MBR, I was looking into serial sniffers, but the ptys broke the parity check. However, his sniffer was not helping me, as I'm using a CP2102, and not a FT232. So I wrote my own sniffer, by following this, and now I have one that can record file I/O on arbitrary files: I called it tracie.
Related
I have two codes : Prog1 writes data to serial port Prog 2 reads data from serial port.
How do I run these programs simultaneously(pipeline) (on the same pc-loopback).
As far as i know: prog1 | prog2 will connect the two programs but through stdin and stdout.
I want to connect them through the output of prog1 I've sent on serial port and receive it as input for prog2.
Your explanation
prog1 generates binary data and send it to the RS232C(USB-serial) port.
prog2 receives the binary data from RS232C(USB-serial) port.
It sounds to me that the prog1 output is already connected to the input of prog2,
without using pipe...
BTW, pipe is used for inter process communication using system memory.
pipe() system call returns two files descriptors, one channel with a
"read end" and the other channels with a "write end".
Then, you can communicate the data between two processes by using these descriptors respectively.
Please refer pipe(2) man page for the detail.
In order to do the loop back using serial port, I think you can not use pipe.
you have to use the descriptor by opening the serial port device file(/dev/ttyXX),
instead of the descriptor generated by pipe.
There are various way to achieve loopback.
RS232C TX and RX signal is physically connected.
In this case, just open the device file of serial port (/dev/ttyxx).
use read(), write() ioctl to exchange the data.
Use the hardware dependent loop back function.
Some hardware can support loop back function for test purpose.
you might be able to use this via ioctl(TIOCMBIS) with TIOCM_LOOP flag set.
Refer tty_ioctl().
Using echo function of stty. Refer stty man page.
Hope this would help you.
I am using my open source Serial-to-IP converter (Serial Network Interface, SNI) to communicate with headless Slackware server using ttyS0 console. SNI during connection sends back to Telnet command set ff fb 01 ff fb 03. This turns Telnet from Line mode to Char mode. But i notice that Telnet replaces (adds) my CR press (0x0d) to two chars 0x0d 0x00. The Slackware's TTY itself is immune to this, but some commands i run under it, at least cat and mcedit, are not. So i try to do the following:
Insert filter in my SNI so it just throw away NULLs at direction IP->Serial. It works, but it quite ugly, because no binary transfer will be allowed anymore.
Modify Telnet's source code to remove addind NULL to CR:
inetutils-1.9.4/telnet/telnet.c from line 2294:
case '\r':
if (!crlf)
{
NETADD ('\r'); <-- added
// NET2ADD ('\r', '\0'); <-- removed
}
else
{
NET2ADD ('\r', '\n');
}
bol = flushline = 1;
break;
It also work, so i ensure i am on the right way catching the problem.
But it is also ugly due to multiple reasons, include that it is quite not portable, and obligate me to have my own non-standard telnet binary.
So the question is: It is possible to command to Telnet not to modify my CRs? (looking at this source code piece, it is not possible at all with original code, but i am sure i miss something, and this should be possible without modify source code). Note: Replace 0x0d to 0x0d, 0x0a (this is present in code) will not work with Linux TTY: it interprets this as two CRs.
And sub-question is: Where is 0x0d+0x00 used at all? I do not know any hardware device, teletype machine, etc., and any TTY, where NULL after CR used, at all. Thanks.
So i take some investigation to see how this should be solved correctly, in terms of not to abuse the software, and to find the correct tools, or clearly show there are no such tools currently.
First of all, i show why my setup is exactly as it is, and why it uses (or not uses) standard tools.
My goal is very common and essential to Linux world (as i think before). Because Linux is network operating system, Linux box should be fully configurable via network (so can/should be fully useful when headless, i.e. without display monitor). But at time when Lilo/Grub starts, there is no network. Only serial ports are able at this time (and Lilo/Grub supports it). Why it is important to remote control at this time? Just because you can (remotely) compile new custom kernel for your Slackware box, and want to test it, adding as 2nd option in Lilo list, and want to return to original kernel even when remote machine stuck when booting, so no way to remotely edit/agjust Lilo options.
But serial console is really much more powerful tool for Linux machine.
It shows boot messages and shut down messages which impossible to see via ssh, due to network is not initialized at these moments. (And remember, we do not have display).
It lets you to (suddenly or intentionally) drop all network interfaces without a fear to lost your machine, when it is many miles from you.
Note: Serial console will not work out-of-box, but can be configured in quite well known standard ways, and described in many places. Example is http://docs.slackware.com/howtos:general_admin:serial_console
Note: There is problem that serial port should be BIOS-recognized, i.e., fully onboard. USB and PCIE devices will never work at stage when Lilo/Grub boots. Fortunately there is a good news. Hardware serial ports now (slowly) return to motherboards. I test new modern Asus J1900I-C for my server, it have two rs232 ports, and it all work fine.
To use serial console benefits remotely, some sort of SNI, Serial Network Interface, should be used (and it is the only way, AFAIK). SNI typically contains (simplest possible) TCP listening server, and raw IP-to-serial duplex translator. I start study and using SNIs when Lantronix X-Port was invented. It was in 2006, and was working fine with Slackware box. (I can't remember if there was \r\0 problems or not with XPort, but let's continue). XPort was too expensive, and was replaced with my own open source SNI immediately when Wiznet W5100 was issued. (Really, there is other, essential problem, why i need to replace Lantronix. It can't press Reset or Power buttons on motherboard, while my device can, and now server is absolutely under full remote control even after kernel panic; but it is offtopic here). Also nowadays there are many cheap no-named SNIs at online shops; i do not test these.
All these times i was using telnet to connect to SNI. The main reason was that XPort docs have examples of that. And, most times it working; i can't say that \r\0 problem stops my work. Most times it is not noticeable at all (say, mc commander is resistent to NULL 's). But mcedit fear the NULLs. And recently i start to catch the problem, so this question arrives as an result.
(now please re-read from begin of thread).
And now i try to answer to my own question. All these times i was misuse the software. Telnet was not written for human communication; instead, Telnet (suddenly) use Telnet protocol, not Raw protocol; and \r\0s are probably part of protocol. (probably, because RFC says nothing about requirement of NULLs anywhere).
While Telnet and Raw protocols may looks quite close, they are not match, so sometimes should work but sometimes not.
Using bruteforce and recompiling binary, i show that Telnet can be very easily turned to Raw mode; but it is non-standard software after that, and there is no chance that it can be pushed to worldwide repos.
So i search for Raw utilites.
netcat and ncat are not have Char mode, only Line mode, so only pure console possible, no ANSI colors, mc, passwords, etc. Putty is too complex and uses GUI. And... No more utilites i found! This was quite strange and annoying...
Then i try to use initially character-based terminals, minicom and gtkterm. They are both do not allow feed ip:port structure istead of ttyS* name. But there is data translators exist, i try socat. It connects to SNI server and creates virtual serial port; then character terminal software connects to that port.
And that, finally, work. Whoa.
But very many disadvantages in this chain; complex, hard to remember commands, can't be written in one line; when SNI drops TCP connection, it is impossible to see/catch that; a LOT of garbage issued by socat into virtual port at connection time (457 packets i count!). Here are the commands for brave people who may find my work useful.
sudo socat pty,link=/dev/ttyMYPORT,raw tcp:10.1.1.11:10001 &
then
sudo chmod a+rw /dev/ttyMYPORT; gtkterm -p /dev/ttyMYPORT
or
sudo chmod a+rw /dev/ttyMYPORT; minicom -o --color=on -D /dev/ttyMYPORT
And conclusion, i do not found any simple software nowadays, that can be directly used to communicate in human raw character mode with serial port via network bridge. (Please suggest something). Thanks.
I believe the solution you are looking for is a few lines higher:
if (!crlf)
If we set crlf = true, then we would get \r\n instead of \r\0. That's probably going to work for cat and friends.
Looking through the telnet source, it appears crlf is some sort of "toggle" option. Looking an man telnet and a bit of Googling, it appears you can run something like this:
$ telnet
telnet> toggle crlf
...and you'll get \r\n sent from then on.
Does anyone know of a good way to get a bi-directional dump of MIDI SysEx data on Linux? (between a Yamaha PSR-E413 MIDI keyboard and a copy of the Yamaha MusicSoft Downloader running in Wine)
I'd like to reverse-engineer the protocol used to copy MIDI files to and from my keyboard's internal memory and, to do that, I need to do some recording of valid exchanges between the two.
The utility does work in Wine (with a little nudging) but I don't want to have to rely on a cheap, un-scriptable app in Wine when I could be using a FUSE filesystem.
Here's the current state of things:
My keyboard connects to my PC via a built-in USB-MIDI bridge. USB dumpers/snoopers are a possibility, but I'd prefer to avoid them if possible. I don't want to have to decode yet another layer of protocol encoding before I even get started.
I run only Linux. However, if there really is no other option than a Windows-based dumper/snooper, I can try getting the USB 1.1 pass-through working on my WinXP VirtualBox VM.
I run bare ALSA for my audio system with dmix for waveform audio mixing.
If a sound server is necessary, I'm willing to experiment with JACK.
No PulseAudio please. It took long enough to excise it from my system.
If the process involves ALSA MIDI routing:
a virtual pass-through device I can select from inside the Downloader is preferred because it often only appears in an ALSA patch bay GUI like patchage an instant before it starts communicating with the keyboard.
Neither KMIDIMon nor GMIDIMonitor support snooping bi-directionally as far as I can tell.
virmidi isn't relevant and I haven't managed to get snd-seq-dummy working.
I I suppose I could patch ALSA to get dumps if I really must, but it's really an option of last resort.
The vast majority of my programming experience is in Python, PHP, Javascript, and shell script.
I have almost no experience programming in C.
I've never even seen a glimpse of kernel-mode code.
I'd prefer to keep my system stable and my uptime high.
This question has been unanswered for some time and while I do not have an exact answer to your problem I maybe have something that can push you in the right direction (or maybe others with similar problems).
I had a similar albeit less complex problem when I wanted to sniff the data used to set and read presets on an Akai LPK25 MIDI keyboard. Similar to your setup the software to setup the keyboard can run in Wine but I also had no luck in finding a sniffer setup for Linux.
For the lack of an existing solution I rolled my own using ALSA MIDI routing over virmidi ports. I understand why you see them as useless because without additional software they cannot help at sniffing MIDI traffic.
My solution was programming a MIDI relay/bridge in Java where I read input from a virmidi port, display the data and send it further to the keyboard. The answer from the keyboard (if any) is also read, displayed and finally transmitted back to the virmidi port. The application in Wine can be setup to use the virmidi port for communication and in theory this process is completely transparent (except for potential latency issues). The application is written in a generic way and not hardcoded to my problem.
I was only dealing with SysEx messages of about 20 bytes length so I am not sure how well the software works for sniffing the transfer of large amounts of data. But maybe you can modify it / write your own program following the example.
Sources available here: https://github.com/hiben/MIDISpy
(Java 1.6, ant build file included, source is under BSD license)
I like using aseqdump for that.
http://www.linuxcommand.org/man_pages/aseqdump1.html
You could use virtual midi devices for this purpose. So you have to load snd_seq_dummy so that it creates at least two ports:
$ sudo modprobe -r snd_seq_dummy
$ sudo modprobe snd_seq_dummy ports=1 duplex=1
Then you should have a device named Midi through:
$ aconnect -i -o -l
client 0: 'System' [type=kernel]
0 'Timer '
1 'Announce '
client 14: 'Midi Through' [type=kernel]
0 'Midi Through Port-0:A'
1 'Midi Through Port-0:B'
client 131: 'VMPK Input' [type=user,pid=50369]
0 'in '
client 132: 'VMPK Output' [type=user,pid=50369]
0 'out '
I will take the port and device numbers form this example. You have to inspect them yourself according to your setup.
Now you plug your favourate MIDI Device to the Midi Through ports:
$ aconnect 132:0 14:0
$ aconnect 14:0 131:0
At this time you have a connection where you can spy on both devices at the same time. You could use aseqdump to spy the MIDI conversation. There are different possibilities. I suggest to spy the connection between the loopback devices and the real device. This allows you for rawmidi connections to the loopback devices.
$ aseqdump -p 14:0,132:0 | tee dump.log
Now everything is set up for use. You just have to be careful about port names in your MIDI application. It should read MIDI data from Midi Through Port-0:B and write data to Midi Through Port-0:B.
Some additional hint: You could use the graphical frontend patchage for connecting and inspecting the MIDI connections via drag and drop. If you do this you will see that every Midi Through port occurs twice once as input and once as output. Both have to be connected in order to make this setup work.
If you want to use GMidiMonitor or some other application you spy on both streams intermixed (without showing the direction) using aconnect suppose 129:0 is the Midi Monitor port :
$ aconnect 14:0 129:0
$ aconnect 132:0 129:0
If you want to have exact direction information you could add another GMidiMonitor instance that you connect only to one of the ports. The missing messages come from the other port.
What about using gmidimonitor? See http://home.gna.org/gmidimonitor/
I am a newbie to developing drivers for Linux ... . I am developing a SMS-driver (AT commands over serial port to modem) using TTY for accessing the serial port. The driver is written in C.
In the design messages from modem to driver can be triggered by two events:
1) Status as respond to AT commands issued by driver (i.e. expected messages)
2) Indication of new SMS (i.e. unexpected messages)
I am planning on two threads - one for writing to TTY and one for reading from TTY. Is it possible to configure TTY so that my read-thread wakes-up on incoming chars (i.e. read-thread is event-triggered and not based on polling)?
Best Regards,
Witek
I don't think you really want two threads. Typical program flow (write AT command, check response etc ...) will be easier to write and debug in a monothreaded program.
Waiting for chars can be made with select() call. The tty layer is mostly configured through
the tcsetattr, tcgetattr and friends system call. With this call you can configure wether you want to be interrupted on new line or on each char for example. See man termios for the manpage. The two big options are wether you want the special characters like EOF, EOL Ctrl-C etc... to be treated has data (raw mode) or be interpreted by the tty layer (canonical mode).
See the part on select in the serial programming guide, or the select manpage for more info
This seems like a simple question, but it is difficult to search for. I need to interface with a device over the serial port. In the event my program (or another) does not finish writing a command to the device, how do I ensure the next run of the program can successfully send a command?
Example:
The foo program runs and begins writing "A_VERY_LONG_COMMAND"
The user terminates the program, but the program has only written, "A_VERY"
The user runs the program again, and the command is resent. Except, the device sees "A_VERYA_VERY_LONG_COMMAND," which isn't what we want.
Is there any way to make this more deterministic? Serial port programming feels very out-of-control due to issues like this.
The required method depends on the device.
Serial ports have additional control signal lines as well as the serial data line; perhaps one of them will reset the device's input. I've never done serial port programming but I think ioctl() handles this.
There may be a single byte which will reset, e.g. some control character.
There might be a timing-based signal, e.g. Hayes command set modems use “pause +++ pause”.
It might just reset after not receiving a complete command after a fixed time.
It might be useful to know whether the device was originally intended to support interactive use (serial terminal), control by a program, or both.
I would guess that if you call write("A_VERY_LONG_COMMAND"), and then the user hits Ctrl+C while the bytes are going out on the line, the driver layer should finish sending the full buffer. And if the user interrupts in the middle of the call, the driver layer will probably just ignore the whole thing.
Just in case, when you open a new COM port, it's always wise to clear the port.
Do you have control over the device end? It might make sense to implement a timeout to make the device ignore unfinished or otherwise corrupt packets.
The embedded device should be implemented such that you can either send an abort/clear/break character that will dump the contents of its command buffer and give you a clean slate on your client app startup.
Or else it should provide a software reset character which will reset the command buffer and all state.
Or else it so be designed so that you can send a command termination (perhaps a newline, etc, depending on command protocol) and possibly have an error generated on the parsing of a garbled partial command that was in its buffer, query/clear the error, and then be good to go.
It wouldn't be a bad idea upon connection of your client program to send some health/status/error query repeatedly until you get a sound response, and only then commence sending configuration or operation commands. Unless you can via a query determine that the device was left in a suitable state, you probably want to assume nothing and configure it from scratch, after a configuration reset if available.