Does anyone know of a good way to get a bi-directional dump of MIDI SysEx data on Linux? (between a Yamaha PSR-E413 MIDI keyboard and a copy of the Yamaha MusicSoft Downloader running in Wine)
I'd like to reverse-engineer the protocol used to copy MIDI files to and from my keyboard's internal memory and, to do that, I need to do some recording of valid exchanges between the two.
The utility does work in Wine (with a little nudging) but I don't want to have to rely on a cheap, un-scriptable app in Wine when I could be using a FUSE filesystem.
Here's the current state of things:
My keyboard connects to my PC via a built-in USB-MIDI bridge. USB dumpers/snoopers are a possibility, but I'd prefer to avoid them if possible. I don't want to have to decode yet another layer of protocol encoding before I even get started.
I run only Linux. However, if there really is no other option than a Windows-based dumper/snooper, I can try getting the USB 1.1 pass-through working on my WinXP VirtualBox VM.
I run bare ALSA for my audio system with dmix for waveform audio mixing.
If a sound server is necessary, I'm willing to experiment with JACK.
No PulseAudio please. It took long enough to excise it from my system.
If the process involves ALSA MIDI routing:
a virtual pass-through device I can select from inside the Downloader is preferred because it often only appears in an ALSA patch bay GUI like patchage an instant before it starts communicating with the keyboard.
Neither KMIDIMon nor GMIDIMonitor support snooping bi-directionally as far as I can tell.
virmidi isn't relevant and I haven't managed to get snd-seq-dummy working.
I I suppose I could patch ALSA to get dumps if I really must, but it's really an option of last resort.
The vast majority of my programming experience is in Python, PHP, Javascript, and shell script.
I have almost no experience programming in C.
I've never even seen a glimpse of kernel-mode code.
I'd prefer to keep my system stable and my uptime high.
This question has been unanswered for some time and while I do not have an exact answer to your problem I maybe have something that can push you in the right direction (or maybe others with similar problems).
I had a similar albeit less complex problem when I wanted to sniff the data used to set and read presets on an Akai LPK25 MIDI keyboard. Similar to your setup the software to setup the keyboard can run in Wine but I also had no luck in finding a sniffer setup for Linux.
For the lack of an existing solution I rolled my own using ALSA MIDI routing over virmidi ports. I understand why you see them as useless because without additional software they cannot help at sniffing MIDI traffic.
My solution was programming a MIDI relay/bridge in Java where I read input from a virmidi port, display the data and send it further to the keyboard. The answer from the keyboard (if any) is also read, displayed and finally transmitted back to the virmidi port. The application in Wine can be setup to use the virmidi port for communication and in theory this process is completely transparent (except for potential latency issues). The application is written in a generic way and not hardcoded to my problem.
I was only dealing with SysEx messages of about 20 bytes length so I am not sure how well the software works for sniffing the transfer of large amounts of data. But maybe you can modify it / write your own program following the example.
Sources available here: https://github.com/hiben/MIDISpy
(Java 1.6, ant build file included, source is under BSD license)
I like using aseqdump for that.
http://www.linuxcommand.org/man_pages/aseqdump1.html
You could use virtual midi devices for this purpose. So you have to load snd_seq_dummy so that it creates at least two ports:
$ sudo modprobe -r snd_seq_dummy
$ sudo modprobe snd_seq_dummy ports=1 duplex=1
Then you should have a device named Midi through:
$ aconnect -i -o -l
client 0: 'System' [type=kernel]
0 'Timer '
1 'Announce '
client 14: 'Midi Through' [type=kernel]
0 'Midi Through Port-0:A'
1 'Midi Through Port-0:B'
client 131: 'VMPK Input' [type=user,pid=50369]
0 'in '
client 132: 'VMPK Output' [type=user,pid=50369]
0 'out '
I will take the port and device numbers form this example. You have to inspect them yourself according to your setup.
Now you plug your favourate MIDI Device to the Midi Through ports:
$ aconnect 132:0 14:0
$ aconnect 14:0 131:0
At this time you have a connection where you can spy on both devices at the same time. You could use aseqdump to spy the MIDI conversation. There are different possibilities. I suggest to spy the connection between the loopback devices and the real device. This allows you for rawmidi connections to the loopback devices.
$ aseqdump -p 14:0,132:0 | tee dump.log
Now everything is set up for use. You just have to be careful about port names in your MIDI application. It should read MIDI data from Midi Through Port-0:B and write data to Midi Through Port-0:B.
Some additional hint: You could use the graphical frontend patchage for connecting and inspecting the MIDI connections via drag and drop. If you do this you will see that every Midi Through port occurs twice once as input and once as output. Both have to be connected in order to make this setup work.
If you want to use GMidiMonitor or some other application you spy on both streams intermixed (without showing the direction) using aconnect suppose 129:0 is the Midi Monitor port :
$ aconnect 14:0 129:0
$ aconnect 132:0 129:0
If you want to have exact direction information you could add another GMidiMonitor instance that you connect only to one of the ports. The missing messages come from the other port.
What about using gmidimonitor? See http://home.gna.org/gmidimonitor/
Related
Using BlueZ, which
is the official Linux Bluetooth stack
I'd like to know which of the below two methods are better suited for detecting a device's presence in the nearby.
To be more exact, I want to periodically scan for a Bluetooth device (not BLE => no advertisement packets are sent).
I found two ways of detecting it:
1.) Using l2ping
# l2ping BTMAC
2.) Using hcitool
# hcitool name BTMAC
Both approaches working.
I'd like to know, which approach would drain more battery of the scanned device?
Looking at solution #1 (l2ping's source):
It uses a standard socket connect call to connect to the remote device, then uses the send command to send data to it:
send(sk, send_buf, L2CAP_CMD_HDR_SIZE + size, 0)
Now, L2CAP_CMD_HDR_SIZE is 4, and default size is 44, so altogether 48 bytes are sent, and received back with L2CAP_ECHO_REQ.
For hcitool I just have found the entrypoint:
int hci_read_remote_name(int dd, const bdaddr_t *bdaddr, int len, char *name, int to);
My questions:
which of these approaches are better (less power-consuming) for the remote device? If there is any difference at all.
shall I reduce the l2ping's size? What shall be the minimum?
is my assumption correct that hci_read_remote_name also connects to the remote device and sends some kind of request to it for getting back its name?
To answer your questions:-
which of these approaches are better (less power-consuming) for the remote device? If there is any difference at all.
l2ping BTMAC is the more suitable command purely because this is what it is meant to do. While "hcitool name BTMAC" is used to get the remote device's name, "l2ping" is used to detect its presence which is what you want to achieve. The difference in power consumption is really minimal, but if there is any then l2ping should be less power consuming.
shall I reduce the l2ping's size? What shall be the minimum?
If changing the l2ping size requires modifying the source code then I recommend leaving it the same. By leaving it the same you are using the same command that has been used countless times and the same command that was used to qualify the BlueZ stack. This way there's less chance for error and any change would not result in noticeable performance or power improvements.
is my assumption correct that hci_read_remote_name also connects to the remote device and sends some kind of request to it for getting back its name?
Yes your assumption is correct. According the Bluetooth Specification v5.2, Vol 4, Part E, Section 7.1.19 Remote Name Request Command:
If no connection exists between the local device and the device
corresponding to the BD_ADDR, a temporary Link Layer connection will
be established to obtain the LMP features and name of the remote
device.
I hope this helps.
I am using my open source Serial-to-IP converter (Serial Network Interface, SNI) to communicate with headless Slackware server using ttyS0 console. SNI during connection sends back to Telnet command set ff fb 01 ff fb 03. This turns Telnet from Line mode to Char mode. But i notice that Telnet replaces (adds) my CR press (0x0d) to two chars 0x0d 0x00. The Slackware's TTY itself is immune to this, but some commands i run under it, at least cat and mcedit, are not. So i try to do the following:
Insert filter in my SNI so it just throw away NULLs at direction IP->Serial. It works, but it quite ugly, because no binary transfer will be allowed anymore.
Modify Telnet's source code to remove addind NULL to CR:
inetutils-1.9.4/telnet/telnet.c from line 2294:
case '\r':
if (!crlf)
{
NETADD ('\r'); <-- added
// NET2ADD ('\r', '\0'); <-- removed
}
else
{
NET2ADD ('\r', '\n');
}
bol = flushline = 1;
break;
It also work, so i ensure i am on the right way catching the problem.
But it is also ugly due to multiple reasons, include that it is quite not portable, and obligate me to have my own non-standard telnet binary.
So the question is: It is possible to command to Telnet not to modify my CRs? (looking at this source code piece, it is not possible at all with original code, but i am sure i miss something, and this should be possible without modify source code). Note: Replace 0x0d to 0x0d, 0x0a (this is present in code) will not work with Linux TTY: it interprets this as two CRs.
And sub-question is: Where is 0x0d+0x00 used at all? I do not know any hardware device, teletype machine, etc., and any TTY, where NULL after CR used, at all. Thanks.
So i take some investigation to see how this should be solved correctly, in terms of not to abuse the software, and to find the correct tools, or clearly show there are no such tools currently.
First of all, i show why my setup is exactly as it is, and why it uses (or not uses) standard tools.
My goal is very common and essential to Linux world (as i think before). Because Linux is network operating system, Linux box should be fully configurable via network (so can/should be fully useful when headless, i.e. without display monitor). But at time when Lilo/Grub starts, there is no network. Only serial ports are able at this time (and Lilo/Grub supports it). Why it is important to remote control at this time? Just because you can (remotely) compile new custom kernel for your Slackware box, and want to test it, adding as 2nd option in Lilo list, and want to return to original kernel even when remote machine stuck when booting, so no way to remotely edit/agjust Lilo options.
But serial console is really much more powerful tool for Linux machine.
It shows boot messages and shut down messages which impossible to see via ssh, due to network is not initialized at these moments. (And remember, we do not have display).
It lets you to (suddenly or intentionally) drop all network interfaces without a fear to lost your machine, when it is many miles from you.
Note: Serial console will not work out-of-box, but can be configured in quite well known standard ways, and described in many places. Example is http://docs.slackware.com/howtos:general_admin:serial_console
Note: There is problem that serial port should be BIOS-recognized, i.e., fully onboard. USB and PCIE devices will never work at stage when Lilo/Grub boots. Fortunately there is a good news. Hardware serial ports now (slowly) return to motherboards. I test new modern Asus J1900I-C for my server, it have two rs232 ports, and it all work fine.
To use serial console benefits remotely, some sort of SNI, Serial Network Interface, should be used (and it is the only way, AFAIK). SNI typically contains (simplest possible) TCP listening server, and raw IP-to-serial duplex translator. I start study and using SNIs when Lantronix X-Port was invented. It was in 2006, and was working fine with Slackware box. (I can't remember if there was \r\0 problems or not with XPort, but let's continue). XPort was too expensive, and was replaced with my own open source SNI immediately when Wiznet W5100 was issued. (Really, there is other, essential problem, why i need to replace Lantronix. It can't press Reset or Power buttons on motherboard, while my device can, and now server is absolutely under full remote control even after kernel panic; but it is offtopic here). Also nowadays there are many cheap no-named SNIs at online shops; i do not test these.
All these times i was using telnet to connect to SNI. The main reason was that XPort docs have examples of that. And, most times it working; i can't say that \r\0 problem stops my work. Most times it is not noticeable at all (say, mc commander is resistent to NULL 's). But mcedit fear the NULLs. And recently i start to catch the problem, so this question arrives as an result.
(now please re-read from begin of thread).
And now i try to answer to my own question. All these times i was misuse the software. Telnet was not written for human communication; instead, Telnet (suddenly) use Telnet protocol, not Raw protocol; and \r\0s are probably part of protocol. (probably, because RFC says nothing about requirement of NULLs anywhere).
While Telnet and Raw protocols may looks quite close, they are not match, so sometimes should work but sometimes not.
Using bruteforce and recompiling binary, i show that Telnet can be very easily turned to Raw mode; but it is non-standard software after that, and there is no chance that it can be pushed to worldwide repos.
So i search for Raw utilites.
netcat and ncat are not have Char mode, only Line mode, so only pure console possible, no ANSI colors, mc, passwords, etc. Putty is too complex and uses GUI. And... No more utilites i found! This was quite strange and annoying...
Then i try to use initially character-based terminals, minicom and gtkterm. They are both do not allow feed ip:port structure istead of ttyS* name. But there is data translators exist, i try socat. It connects to SNI server and creates virtual serial port; then character terminal software connects to that port.
And that, finally, work. Whoa.
But very many disadvantages in this chain; complex, hard to remember commands, can't be written in one line; when SNI drops TCP connection, it is impossible to see/catch that; a LOT of garbage issued by socat into virtual port at connection time (457 packets i count!). Here are the commands for brave people who may find my work useful.
sudo socat pty,link=/dev/ttyMYPORT,raw tcp:10.1.1.11:10001 &
then
sudo chmod a+rw /dev/ttyMYPORT; gtkterm -p /dev/ttyMYPORT
or
sudo chmod a+rw /dev/ttyMYPORT; minicom -o --color=on -D /dev/ttyMYPORT
And conclusion, i do not found any simple software nowadays, that can be directly used to communicate in human raw character mode with serial port via network bridge. (Please suggest something). Thanks.
I believe the solution you are looking for is a few lines higher:
if (!crlf)
If we set crlf = true, then we would get \r\n instead of \r\0. That's probably going to work for cat and friends.
Looking through the telnet source, it appears crlf is some sort of "toggle" option. Looking an man telnet and a bit of Googling, it appears you can run something like this:
$ telnet
telnet> toggle crlf
...and you'll get \r\n sent from then on.
I have a telephony modem which gives voice to my interfaced application via a serial USB ttyUSB0 in 16bit PCM 8000hz. I am able to capture this data and play with audacity. I want this port to be detected as a sound device in linux (I am on ubuntu). Is it possible? Are there any other options?
I'm guessing you are using a huawei 3G modem or something similar which gives ttyUSB1 for audio. Make sure you have the serial driver binded to it. Then simply pass the port itself as a "file" for input for any program of your choice.You need root access for that.You figured out the audio settings so it must be enough.I have voice calling working in UBUNTU 11.10 with Huawei. So let me know if i can help any further.
Ok, I see it's very old question but answers helped me to get a right direction so I decided to help others.
The one way to achieve (in addition to below) what are you are
looking for is to write dynamic kernel module.
Have it register as a sound device, and check that it has a GSM
module present (which module is it exactly can be recognized in
dmesg, lsmod, or output).
Then establish communication between user space representation as a
sound card and serial usb module.
The other way is to get module that you recognized by dmesg, lsmod and extend its functionality as a sound card.
All are tricky tasks because:
in the first case you have to resolve intermodule communication at the kernel level...... which is, lets say, a little hard even if programmer has a right background in subject.
the second case is hard in that you have to deal with:
USB stack (which is little unpleasant for human beings) and
sound subsystem (which is a little burdensome because of historical issues).
Without being an experienced kernel programmer there are small chances to succeed.
Sorry for the wall of text - TL;DR:
What is the framerate of VNC connection (in frames/sec) - or rather, who determines it: client or server?
Any other suggestions for desktop screen capture - but "correctly timecoded"/ with unjittered framerate (with a stable period); and with possibility to obtain it as uncompressed (or lossless) image sequence?
Briefly - I have a typical problem that I am faced with: I sometimes develop hardware, and want to record a video that shows both commands entered on the PC ('desktop capture'), and responses of the hardware ('live video'). A chunk of an intro follows, before I get to the specific detail(s).
Intro/Context
My strategy, for now, is to use a video camera to record the process of hardware testing (as 'live' video) - and do a desktop capture at the same time. The video camera produces a 29.97 (30) FPS MPEG-2 .AVI video; and I want to get the desktop capture as an image sequence of PNGs at the same frame rate as the video. The idea, then, would be: if the frame rate of the two videos is the same; then I could simply
align the time of start of the desktop capture, with the matching point in the 'live' video
Set up a picture-in-picture, where a scaled down version of the desktop capture is put - as overlay - on top of the 'live' video
(where a portion of the screen on the 'live' video, serves as a visual sync source with the 'desktop capture' overlay)
Export a 'final' combined video, compressed appropriately for the Internet
In principle, I guess one could use a command line tool like ffmpeg for this process; however I would prefer to use a GUI for finding the alignment start point for the two videos.
Eventually, what I also want to achieve, is to preserve maximum quality when exporting the 'final' video: the 'live' video is already compressed when out of the camera, which means additional degradation when it passes through the Theora .ogv codec - which is why I'd like to keep the original videos, and use something like a command line to generate a 'final' video anew, if a different compression/resolution is required. This is also why I like to have the 'desktop capture' video as a PNG sequence (although I guess any uncompressed format would do): I take measures to 'adjust' the desktop, so there aren't many gradients, and lossless encoding (i.e. PNG) would be appropriate.
Desktop capture options
Well, there are many troubles in this process under Ubuntu Lucid, which I currently use (and you can read about some of my ordeals in 10.04: Video overlay/composite editing with Theora ogv - Ubuntu Forums). However, one of the crucial problems is the assumption, that the frame rate of the two incoming videos is equal - in reality, usually the desktop capture is of a lower framerate; and even worse, very often frames are out of sync.
This, then, requires the hassle of sitting in front of a video editor, and manually cutting and editing less-than-a-second clips on frame level - requiring hours of work for what will be in the end a 5 minute video. On the other hand, if the two videos ('live' and 'capture') did have the same framerate and sync: in principle, you wouldn't need more than a couple of minutes for finding the start sync point in a video editor - and the rest of the 'merged' video processing could be handled by a single command line. Which is why, in this post, I would like to focus on the desktop capture part.
As far as I can see, there are only few viable (as opposed to 5 Ways to Screencast Your Linux Desktop) alternatives for desktop capture in Linux / Ubuntu (note, I typically use a laptop as target for desktop capturing):
Have your target PC (laptop) clone the desktop on its VGA output; use a VGA-to-composite or VGA-to-S-video hardware to obtain a video signal from VGA; use video capture card on a different PC to grab video
Use recordMyDesktop on the target PC
Set up a VNC server (vino on Ubuntu; or vncserver) on the target PC to be captured; use VNC capture software (such as vncrec) on a different PC to grab/record the VNC stream (which can, subsequently, be converted to video).
Use ffmpeg with x11grab option
*(use some tool on the target PC, that would do a DMA transfer of a desktop image frame directly - from the graphics card frame buffer memory, to the network adapter memory)
Please note that the usefulness of the above approaches are limited by my context of use: the target PC that I want to capture, typically runs software (utilizing the tested hardware) that moves around massive ammounts of data; best you could say about describing such a system is "barely stable" :) I'd guess this is similar to problems gamers face, when wanting to obtain a video capture of a demanding game. And as soon as I start using something like recordMyDesktop, which also uses quite a bit of resources and wants to capture on the local hard disk - I immediately get severe kernel crashes (often with no vmcore generated).
So, in my context, I typically do assume involvement of a second computer - to run the capture and recording of the 'target' PC desktop. Other than that, the pros and cons I can see so far with the above options, are included below.
(Desktop preparation)
For all of the methods discussed below, I tend to "prepare" the desktop beforehand:
Remove desktop backgrounds and icons
Set the resolution down to 800x600 via System/Preferences/Monitors (gnome-desktop-properties)
Change color depth down to 16 bpp (using xdpyinfo | grep "of root" to check)
... in order to minimize the load on desktop capture software. Note that changing color depth on Ubuntu requires changes to xorg.conf; however, "No xorg.conf (is) found in /etc/X11 (Ubuntu 10.04)" - so you may need to run sudo Xorg -configure first.
In order to keep graphics resource use low, also I usually had compiz disabled - or rather, I'd have 'System/Preferences/Appearance/Visual Effects' set to "None". However, after I tried enabling compiz by setting 'Visual Effects' to "Normal" (which doesn't get saved), I can notice windows on the LCD screen are redrawn much faster; so I keep it like this, also for desktop capture. I find this a bit strange: how could more effects cause a faster screen refresh? It doesn't look like it's due to a proprietary driver (the card is "Intel Corporation N10 Family Integrated Graphics Controller", and no proprietary driver option is given by Ubuntu upon switch to compiz) - although, it could be that all the blurring and effects just cheat my eyes :) ).
Cloning VGA
Well, this is the most expencive option (as it requires additional purchase of not just one, but two pieces of hardware: VGA converter, and video capture card); and applicable mostly to laptops (which have both a screen + additional VGA output - for desktops one may also have to invest in an additional graphics card, or a VGA cloning hardware).
However, it is also the only option that requires no additional software of the target PC whatsoever (and thus uses 0% processing power of the target CPU) - AND also the only one that will give a video with a true, unjittered framerate of 30 fps (as it is performed by separate hardware - although, with the assumption that clock domains misalignment, present between individual hardware pieces, is negligible).
Actually, as I already own something like a capture card, I have already invested in a VGA converter - in expectation that it will eventually allow me to produce final "merged" videos with only 5 mins of looking for alignment point, and a single command line; but I am yet to see whether this process will work as intended. I'm also wandering how possible it will be to capture desktop as uncompressed video # 800x600, 30 fps.
recordMyDesktop
Well, if you run recordMyDesktop without any arguments - it starts first with capturing (what looks like) raw image data, in a folder like /tmp/rMD-session-7247; and after you press Ctrl-C to interrupt it, it will encode this raw image data into an .ogv. Obviously, grabbing large image data on the same hard disk as my test software (which also moves large ammounts of data), is usually a cause for an instacrash :)
Hence, what I tried doing is to setup Samba to share a drive on the network; then on the target PC, I'd connect to this drive - and instruct recordMyDesktop to use this network drive (via gvfs) as its temporary files location:
recordmydesktop --workdir /home/user/.gvfs/test\ on\ 192.168.1.100/capture/ --no-sound --quick-subsampling --fps 30 --overwrite -o capture.ogv
Note that, while this command will use the network location for temporary files (and thus makes it possible for recordMyDesktop to run in parallel with my software) - as soon as you hit Ctrl-C, it will start encoding and saving capture.ogv directly on the local hard drive of the target (though, at that point, I don't really care :) )
First of my nags with recordMyDesktop is that you cannot instruct it to keep the temporary files, and avoid encoding them, on end: you can use Ctrl+Alt+p for pause - or you can hit Ctrl-C quickly after the first one, to cause it to crash; which will then leave the temporary files (if you don't hit Ctrl-C quickly enough the second time, the program will "Cleanning up cache..."). You can then run, say:
recordmydesktop --rescue /home/user/.gvfs/test\ on\ 192.168.1.100/capture/rMD-session-7247/
... in order to convert the raw temporary data. However, more often than not, recordMyDesktop will itself segfault in the midst of performing this "rescue". Although, the reason why I want to keep the temp files, is to have the uncompressed source for the picture-in-picture montage. Note that the "--on-the-fly-encoding" will avoid using temp files altogether - at the expence of using more CPU processing power (which, for me, again is cause for crashes.)
Then, there is the framerate - obviously, you can set requested framerate using the '--fps N' option; however, that is no guarantee that you will actually obtain that framerate; for instance, I'd get:
recordmydesktop --fps 25
...
Saved 2983 frames in a total of 6023 requests
...
... for a capture with my test software running; which means that the actually achieved rate is more like 25*2983/6032 = 12.3632 fps!
Obviously, frames are dropped - and mostly that shows as video playback is too fast. However, if I lower the requested fps to 12 - then according to saved/total reports, I achieve something like 11 fps; and in this case, video playback doesn't look 'sped up'. And I still haven't tried aligning such a capture with a live video - so I have no idea if those frames that actually have been saved, also have an accurate timestamp.
VNC capture
The VNC capture, for me, consists of running a VNC server on the 'target' PC, and running vncrec (twibright edition) on the 'recorder' PC. As VNC server, I use vino, which is "System/Preferences/Remote Desktop (Preferences)". And apparently, even if vino configuration may not be the easiest thing to manage, vino as a server seems not too taxing to the 'target' PC; as I haven't experienced crashes when it runs in parallel with my test software.
On the other hand, when vncrec is capturing on the 'recorder' PC, it also raises a window showing you the 'target' desktop as it is seen in 'realtime'; when there are large updates (i.e. whole windows moving) on the 'target' - one can, quite visibly, see problems with the update/refresh rate on the 'recorder'. But, for only small updates (i.e. just a cursor moving on a static background), things seem OK.
This makes me wonder about one of my primary questions with this post - what is it, that sets the framerate in a VNC connection?
I haven't found a clear answer to this, but from bits and pieces of info (see refs below), I gather that:
The VNC server simply sends changes (screen changes + clicks etc) as fast as it can, when it receives them ; limited by the max network bandwidth that is available to the server
The VNC client receives those change events delayed and jittered by the network connection, and attempts to reconstruct the desktop "video" stream, again as fast as it can
... which means, one cannot state anything in terms of a stable, periodic frame rate (as in video).
As far as vncrec as a client goes, the end videos I get usually are declared as 10 fps, although frames can be rather displaced/jittered (which then requires the cutting in video editors). Note that the vncrec-twibright/README states: "The sample rate of the movie is 10 by default or overriden by VNCREC_MOVIE_FRAMERATE environment variable, or 10 if not specified."; however, the manpage also states "VNCREC_MOVIE_FRAMERATE - Specifies frame rate of the output movie. Has an effect only in -movie mode. Defaults to 10. Try 24 when your transcoder vomits from 10.". And if one looks into "vncrec/sockets.c" source, one can see:
void print_movie_frames_up_to_time(struct timeval tv)
{
static double framerate;
....
memcpy(out, bufoutptr, buffered);
if (appData.record)
{
writeLogHeader (); /* Writes the timestamp */
fwrite (bufoutptr, 1, buffered, vncLog);
}
... which shows that some timestamps are written - but whether those timestamps originate from the "original" 'target' PC, or the 'recorder' one, I cannot tell.
EDIT: thanks to the answer by #kanaka, I checked through vncrec/sockets.c again, and can see that it is the writeLogHeader function itself calling gettimeofday; so the timestamps it writes are local - that is, they originate from the 'recorder' PC (and hence, these timestamps do not accurately describe when the frames originated on the 'target' PC).
In any case, it still seems to me, that the server sends - and vncrec as client receives - whenever; and it is only in the process of encoding a video file from the raw capture afterwards, that some form of a frame rate is set/interpolated.
I'd also like to state that on my 'target' laptop, the wired network connection is broken; so the wireless is my only option to get access to the router and the local network - at far lower speed than the 100MB/s that the router could handle from wired connections. However, if the jitter in captured frames is caused by wrong timestamps due to load on the 'target' PC, I don't think good network bandwidth will help too much.
Finally, as far as VNC goes, there could be other alternatives to try - such as VNCast server (promising, but requires some time to build from source, and is in "early experimental version"); or MultiVNC (although, it just seems like a client/viewer, without options for recording).
ffmpeg with x11grab
Haven't played with this much, but, I've tried it in connection with netcat; this:
# 'target'
ffmpeg -f x11grab -b 8000k -r 30 -s 800x600 -i :0.0 -f rawvideo - | nc 192.168.1.100 5678
# 'recorder'
nc -l 0.0.0.0 5678 > raw.video #
... does capture a file, but ffplay cannot read the captured file properly; while:
# 'target'
ffmpeg -f x11grab -b 500k -r 30 -s 800x600 -i :0.0 -f yuv4mpegpipe -pix_fmt yuv444p - | nc 192.168.1.100 5678
# 'recorder'
nc -l 0.0.0.0 5678 | ffmpeg -i - /path/to/samplimg%03d.png
does produce .png images - but with compression artifacts (result of the compression involved with yuv4mpegpipe, I guess).
Thus, I'm not liking ffmpeg+x11grab too much currently - but maybe I simply don't know how to set it up for my needs.
*( graphics card -> DMA -> network )
I am, admittedly, not sure something like this exists - in fact, I would wager it doesn't :) And I'm no expert here, but I speculate:
if DMA memory transfer can be initiated from the graphics card (or its buffer that keeps the current desktop bitmap) as source, and the network adapter as destination - then in principle, it should be possible to obtain an uncompressed desktop capture with a correct (and decent) framerate. The point in using DMA transfer would be, of course, to relieve the processor from the task of copying the desktop image to the network interface (and thus, reduce the influence the capturing software can have on the processes running on the 'target' PC - especially those dealing with RAM or hard-disk).
A suggestion like this, of course, assumes that: there are massive ammounts of network bandwidth (for 800x600, 30 fps at least 800*600*3*30 = 43200000 bps = 42 MiB/s, which should be OK for local 100 MB/s networks); plenty of hard disk on the other PC that does the 'recording' - and finally, software that can afterwards read that raw data, and generate image sequences or videos based on it :)
The bandwidth and hard disk demands I could live with - as long as there is guarantee both for a stable framerate and uncompressed data; which is why I'd love to hear if something like this already exists.
-- -- -- -- --
Well, I guess that was it - as brief as I could put it :) Any suggestions for tools - or process(es), that can result with a desktop capture
in uncompressed format (ultimately convertible to uncompressed/lossless PNG image sequence), and
with a "correctly timecoded", stable framerate
..., that will ultimately lend itself to 'easy', single command-line processing for generating 'picture-in-picture' overlay videos - will be greatly appreciated!
Thanks in advance for any comments,
Cheers!
References
Experiences Producing a Screencast on Linux for CryptoTE - idlebox.net
The VideoLAN Forums • View topic - VNC Client input support (like screen://)
VNCServer throttles user inpt for slow client - Kyprianou, Mark - com.realvnc.vnc-list - MarkMail
Linux FAQ - X Windows: How do I Display and Control a Remote Desktop using VNC
How much bandwidth does VNC require? RealVNC - Frequently asked questions
x11vnc: a VNC server for real X displays
HowtoRecordVNC (an X11 session) - Debian Wiki
Alternative To gtk-RecordMyDesktop in Ubuntu
(Ffmpeg-user) How do I use pipes in ffmpeg
(ffmpeg-devel) (PATCH) Fix segfault in x11grab when drawing Cursor on Xservers that don't support the XFixes extension
You should get a badge for such a long well though out question. ;-)
In answer to your primary question, VNC uses the RFB protocol which is a remote frame buffer protocol (thus the acronym) not a streaming video protocol. The VNC client sends a FrameBufferUpdateRequest message to the server which contains a viewport region that the client is interested in and an incremental flag. If the incremental flag is not set then the server will respond with a FrameBufferUpdate message that contains the content of the region requested. If the incremental flag is set then the server may respond with a FrameBufferUpdate message that contains whatever parts of the region requested that have changed since the last time the client was sent that region.
The definition of how requests and updates interact is not crisply defined. The server won't necessarily respond to every request with an update if nothing has changed. If the server has multiple requests queued from the client it is also allowed to send a single update in response. In addition, the client really needs to be able to respond to an asynchronous update message from the server (not in response to a request) otherwise the client will fall out of sync (because RFB is not a framed protocol).
Often clients are simply implemented to send incremental update requests for the entire frame buffer viewport at a periodic interval and handle any server update messages as they arrive (i.e. no attempt is made to tie requests and updates together).
Here is a description of FrameBufferUpdateRequest messages.
I'm debugging communications with a serial device, and I need to see all the data flowing both directions.
It seems like this should be easy on Linux, where the serial port is represented by a file. Is there some way that I can do a sort of "bi-directional tee", where I tell my program to connect to a pipe that copies the data to a file and also shuffles it to/from the actual serial port device?
I think I might even know how to write such a beast, but it seems non-trivial, especially to get all of the ioctls passed through for port configuration, etc.
Has anyone already built such a thing? It seems too useful (for people debugging serial device drivers) not to exist already.
strace is very useful for this. You have a visualisation of all ioctl calls, with the corresponding structure decoded. The following options seems particularly useful in your case:
-e read=set
Perform a full hexadecimal and ASCII dump of all the data read from
file descriptors listed in the
specified set. For example, to see all
input activity on file descriptors 3
and 5 use -e read=3,5. Note that this
is independent from the normal tracing
of the read(2) system call which is
controlled by the option -e
trace=read.
-e write=set
Perform a full hexadecimal and ASCII
dump of all the data written to file
descriptors listed in the specified
set. For example, to see all output
activity on file descriptors 3 and 5
use -e write=3,5. Note that this is
independent from the normal tracing of
the write(2) system call which is
controlled by the option -e
trace=write.
I have found pyserial to be quite usable, so if you're into Python it shouldn't be too hard to write such a thing.
A simple method would be to write an application which opened
the master side of a pty and the tty under test. You would then
pass your tty application the slave side of the pty as the 'tty device'.
You would have to monitor the pty attributes with tcgetattr() on the pty
master and call tcsetattr() on the real tty, if the attributes changed.
The rest would be a simple select() on both fd's copying data bi-directionally and copying it to a log.
I looked at a lot of serial sniffers. All of them are based on the idea of making a virtual serial port and sniff data from that port. However, any baud/parity/flow changes will break connection.
So, I wrote my own sniffer :). Most of the serial ports now are just USB-to-serial converters. My sniffer collects data from USB through debugfs, parse it and output to the console. Also any baudrate changes, flow control, line events, and serial errors are also recorded. The project is in the early stage of development and for now, only FTDI is supported.
http://code.google.com/p/uscmon/
Much like #MBR, I was looking into serial sniffers, but the ptys broke the parity check. However, his sniffer was not helping me, as I'm using a CP2102, and not a FT232. So I wrote my own sniffer, by following this, and now I have one that can record file I/O on arbitrary files: I called it tracie.