I'm having a problem with LIRC breaking audio on the OS scale after firing a command.
For example, I'd do:
irsend send_once Samsung_BN59-01224C KEY_VOLUMEUP --count=5
and afterwards, play an audio file, and the program governing that file would seize up and not play any sound. Same goes for a script I've written that uses the pygame library for python.
What's worse is that LIRC also stops firing correctly after this bug occurs. I can see infrared light being shot out of the diode, but there might be something off with the timing.
This happens both ways, so, after playing an audio file, LIRC will stop working but further playing of audio is possible.
The following extremely rarely but sometimes I'm able to play audio after LIRC finishes a command, and the result is heavily pitched down version of the original sound that cuts out after around a second or so.
Tested with different remotes, same results occur. I'm not sure if the fix that a user proposed in this thread could cause this (https://github.com/raspberrypi/linux/issues/2993) but I'm putting it out there that I used it, since unmodified LIRC has problems with both the receiver and transmitter turned on in /boot/config.txt. The rest of my installation is standard.
Fixed this by reverting the fix I posted in the last paragraph. Apparently using PWM for Infrared causes issues with onboard audio on raspbian. I commented out the lines responsible for the receiver and left the gpio-ir-tx option uncommented. Works fine with just the transmitter on.
Related
I have a very interesting problem.
I am running custom movie player based on NDK/C++/CMake toolchain that opens streaming URL (mp4, H.264 & stereo audio). In order to restart from given position, player opens stream, buffers frames to some length and then seeks to new position and start decoding and playing. This works fine all the times except if we power-cycle the device and follow the same steps.
This was reproduced on few version of the software (plugin build against android-22..26) and hardware (LG G6, G5 and LeEco). This issue does not happen if you keep app open for 10 mins.
I am looking for possible areas of concern. I have played with decode logic (it is based on the approach described as synchronous processing using buffers).
Edit - More Information (4/23)
I modified player to pick a stream and then played only video instead of video+audio. This resulted in constant starvation resulting in buffering. This appears to have changed across android version (no fix data here). I do believe that I am running into decoder starvation. Previously, I had set timeouts of 0 for both AMediaCodec_dequeueInputBuffer and AMediaCodec_dequeueOutputBuffer, which I changed on input side to 1000 and 10000 but does not make much difference.
My player is based on NDK/C++ interface to MediaCodec, CMake build passes -DANDROID_ABI="armeabi-v7a with NEON" and -DANDROID_NATIVE_API_LEVEL="android-22" \ and C++_static.
Anyone can share what timeouts they have used and found success with it or anything that would help avoid starvation or resulting buffering?
This is solved for now. Starvation was not caused from decoding perspective but images were consumed in faster pace as clock value returned were not in sync. I was using clock_gettime method with CLOCK_MONOTONIC clock id, which is recommended way but it was always faster for first 5-10 mins of restarting device. This device only had Wi-Fi connection. Changing clock id to CLOCK_REALTIME ensures correct presentation of images and no starvation.
For a computer vision project that I am working on I need to grab images using a Logitech C920 webcam. I am using OpenCV's VideoCapture to do that, but the problem that I am facing is that the image that I take at a certain moment does not show the latest thing that the camera sees. That is, if I take an image at timestamp t, it shows what the camera saw at timestamp (t - delta), so to say.
I did this by writing a program that increments a counter and shows it on the screen. I pointed the camera at the screen and let it record. When the counter reached a certain value, say 10000, it would grab an image and save it with the filename "counter_value.png" (e.g. 10000.png). That way I was able to compare the current value of the counter with the current value seen by the camera. I noticed that most of the time the delay is about 4-5 frames, but it is not a fixed value.
I saw similar posts about this issue, but none of them really helped. Some people recommended putting the frame grabbing routine into a separate thread and updating a "current_frame" Mat variable. I tried that, but for some reason the issue is still present. Someone else mentioned that the camera worked well on Windows (but I need to use Linux, though). I tried running the same code on Windows and indeed the delay was only about 1 frame (which might as well be that the camera did not see the counter because the screen did not update fast enough).
I then decided to run a simple webcam viewer based only on V4L2 code, thinking that the issue might be coming from OpenCV. I again experienced the same delay, which makes me believe that the driver is using some sort of buffer to cache the images.
I am new to V4L2 and I really need to solve this problem as soon as possible, so my questions to you guys are:
Has anyone found a solution for getting the latest image using V4L2 (and maybe OpenCV)?
If there is no way to solve it using V4L2, does anyone know another alternative to fixing this issue on Linux?
Regards,
Mihai
It looks like that there will be always a delay between the VideoCapture::grab() call and when the frame is actually taken. This is because of frame buffering that is done at hardware/SO level and you cannot avoid that.
OpenCV provides the VideoCapture::get( CV_CAP_PROP_POS_MEC) ) method to give you the exact time a frame was captured, but this is only possible if the camera supports it.
Recently a problem has been discovered in V4L OpenCV implementation:
http://answers.opencv.org/question/61099/is-it-possible-to-get-frame-timestamps-for-live-streaming-video-frames-on-linux/
And a few days ago a fix has been pulled:
https://github.com/Itseez/opencv/pull/3998
In the end, if you have the right setup, you can know what is the time a frame was taken (and therefore compensate).
It is possible the problem is with the Linux UVC driver, but I have been using Microsoft LifeCam Cinemas for machine vision on Ubuntu 12.04 and 14.04 machines, and have not seen a 4-5 frame delay. I operate them in low light conditions, though, in which case they reduce the frame rate to 7.5 fps.
One other possible culprit is a delay in the webcam depending what format is used. The C920 appears to support H.264 (which few webcams do), so Logitech may have put most effort to make this work well, yet OpenCV appears not to support H.264 on Linux; see this answer for what formats it supports. The same question also has an answer with a kernel hack(!) to fix an issue with the UVC driver.
PS: to check the format actually used in my case, I added
fprintf(stderr, ">>> palette: %d\n", capture->palette);
at this line in the OpenCV code.
I've been playing around with some speech-to-text and text-to-speech systems, and am running into the problem that when the computer makes sounds that it can recognize, it starts taking commands from itself. To avoid this, I'd like a stream of all sounds picked up by the microphone that were not produced by the computer itself.
I see that PulseAudio has an echo cancellation module, but so far I have been unable to distinguish between its output and the raw microphone output: it still contains all the sounds picked up by the microphone that came from the computer speakers. I wonder if the default echo canceller is doing the opposite of what I want (i.e., it removes sounds heard by the microphone from being sent to the speakers).
Any idea how I can do this (preferably with pacmd)? I have thoroughly confused myself trying to specify non-default sources for the echo canceller, and have wandered into loopback modules and other things that are probably irrelevant. I know very little about PulseAudio, haven't found a good introduction to it (I've looked through much of the PulseAudio documentation but didn't see anything relevant), and might just be missing something simple. I feel frustrated that echo cancellation apparently doesn't work, I can't find documentation on it, and I can't find examples of it working from other people.
Thanks in advance for the help!
Other details that might be relevant: I'm running Ubuntu Saucy on a Lenovo Thinkpad T410. I'm using the built-in microphone and speakers (so, I'm pretty sure they're using the same sound card and I won't have clock drift issues). My actual application gets its sound through GStreamer, but GStreamer gets it from PulseAudio, and I don't think GStreamer itself has AEC capabilities. If there's a different way of doing this, I'd gladly switch to that.
Ah, I've got it! Merely loading the echo cancellation plugin isn't enough; you then need to start using it. In particular, it will only cancel echos of sounds passed into it, and if no sounds go through it, nothing will be cancelled. So, open /etc/pulse/default.pa and add the line
load-module module-echo-cancel
towards the bottom (I put it right after the line that loads module-filter-apply). Then, restart the PulseAudio daemon by running (as a non-root user) pulseaudio -k. Next, run pacmd to get a command line interface to PulseAudio, and give it the commands list-sources and list-sinks. Note the indices of the echo canceller in the responses. Edit /etc/pulse/default.pa again, and uncomment the two lines at the end about set-default, replacing the words input and output with the indices of the echo canceller's source and sink. Finally, restart PulseAudio again with pulseaudio -k (again, run as a non-root user).
Now, by default all sounds to be output get sent through the echo canceller before heading to the speakers, and all sounds to be input get pulled from the echo canceller after coming in through the microphone, and things actually work. You can verify that it's working by running pavucontrol and looking at the sound levels on the Input Devices screen (try playing some music and speaking, and note that the echo cancelled input shows normal sound levels when you speak but very low levels (verging on nothing) when you're silent but the music is playing).
This answer mostly comes from this post, which I wish I'd found weeks ago.
I have a program written in C++ that uses RtAudio ( Directsound ) to capture and playback audio at 48kHz samplerate.
The input capture uses a callback option. The callback writes data to a ringbuffer.
The output is a blocking write function in a separate thread that reads from the ringbuffer.
If the input and output devices are the same the audio loops thru perfectly.
Now I want to get audio from device 1 and playback on device 2. Each device has its own sampleclock set to 48kHz but are not in sync. After a couple of seconds the input and output are out of sync.
Is it possible to sync two independent oudio devices?
There are two challenges you face:
getting the two devices to start at the same time.
getting the two devices to stay in sync.
Both of these tasks are difficult. In the pro audio world, #2 is accomplished with special hardware to sync the word-clocks of multiple devices. It can also be done with a high quality video signal. I believe it can also be done with firewire devices, but I'm not sure how that works. In practice, I have used devices with no sync ("wild") and gotten very reasonable sync for up to an hour or two. Depending on what you are trying to do, the sync should not drift more than a few milliseconds over the course of a few minutes. If it does, you can consider your hardware broken (of course, cheap hardware is often broken).
As for #1, I'm not sure this is possible in any reliable sense with directsound. To the extent that it's possible with any audio API, it is difficult at best: both cards have streams that require some time to setup, open and start playing. In general, the solution is to use an API where this time is super low (ASIO, for example). This works reasonably well for applications like video, but I don't know if it really solves the problem in general.
If you really need to solve this problem, you could open both cards, starting to play silence, and use the timing information generated by the cards to establish the delay between putting data into the card and its eventual playback (this will be different for each card and probably each time you run) and use that data to calculate when to start actual playback. I don't know if RTAudio supplies the necessary timing information, but PortAudio does. This document may help.
I am trying to write an application(I'm a gui first timer) for my son, he has autism. There is a video player in the top half and a text entry area in the bottom. When letters are typed sounds are produced to mimic the words in the video.
There have been other posts on this site in regard to playing sounds on key presses, using gstreamer as a system call. I have also tried libcanberra but both seem to have significant delays between sounds. I can write the app in python or C but will likely do at least some of it in C.
I also want to mention that the video portion is being played by gstreamer. I tried to create two instances of gstreamer, to avoid expensive system calls but the audio instance seemed to kill the app when called.
If anyone has any tips on creating faster responding sounds I would really appreciate it.
You can upload a raw audio sample directly to PulseAudio so there will be no decoding and (perhaps save) extra switches by using the following function from Canberra:
http://developer.gnome.org/libcanberra/unstable/libcanberra-canberra.html#ca-context-cache
The next ca_context_play() will use it.
However, the biggest problem you'll encounter with this scenario (with simultaneous video playback) is that the audio device might be configured with large latency with PulseAudio (up to 1/2s or more for normal playback). It may be reasonable to file a bug to libcanberra to support a LOW_LATENCY flag, as it currently doesn't attempt to minimize delay for sound events afaik. That would be great to have.
GStreamer pulsesink could probably get low latency too (it has some properties for that), but I am afraid it won't be as lightweight as libcanberra, and you won't be able to cache a sample for instance. Ideally, GStreamer could also learn to cache samples, or pre-fill PulseAudio...