I have a strange issue with SDL audio. It will randomly become distorted. It's completely random, I can't reliably reproduce it at all. Sometime it starts being distorted seconds after the program starts, other times I've played with the software, let it run over night and it'll still be working in the morning.
It could have to do with SDL, because I can close the mixer with Mix_CloseAudio() and then re-open it with Mix_OpenAudio() and it'll fix the sound... for a while, sometimes a long while, other times it'll mess up soon after. (As Brad pointed out in the comments, this doesn't necessarily mean it's an SDL issue).
More about the system:
stripped down linux system with kernel 2.6.27.5-117.fc10.i686.
libSDL_mixer-1.2.so.0.2.6.
This is a legacy system that always worked in the past, and I've made no changes at all to the audio system part of the software. The system used to run off a CD-rom and all the game's assets were loaded into ram on a filesystem overlay, but I've switched it to running off of a CF card without all the sounds in ram all the time. I've made no changes to the sound drivers either.
Here's an example of the sound distortion https://www.youtube.com/watch?v=zEneMi5mtt0
Here's an example of the same sounds functioning properly https://www.youtube.com/watch?v=nyPZLJ-9gsI
Some observations based on your sound that we can use to debug the issue:
The distortion you're hearing is based on your sound, and is a series of regular clicks.
If the audio is quiet, the distortion/clicks are quiet. Likewise, the problem gets worse as the audio gets louder.
Pitch is correct, so it isn't a sample rate issue.
Once the problem begins, it is consistent.
Basically, what's happening is that the sound card is reading from the circular buffer at a different place than where the software writing to it thinks it is. At some point, there was a buffer underrun causing the play head to get in-front of the read head.
What's supposed to happen is that the playback should reset, causing a momentary drop-out of audio until the buffer is full again. What seems to have happened is that playback continued. Therefore, the play head keeps running over a line between old buffer data and new buffer data, causing the click. This is also why when the sound is quieter, the click/distortion is quieter... the difference between the two PCM samples is less.
I've seen this happen with buggy sound card drivers, and also buggy sound cards. To work around the problem, find a way to increase your buffer size. This increases latency but may help prevent the problem from occurring in the first place.
Related
Is there a way to fully dump the configuration of an Intel HDA-based audio codec, including current hardware state?
The interface at /proc/asound/card0/codec#0 only reflects what's known to the kernel, not the real hardware state.
The PCI configuration space (read with lspci -x) doesn't show much (it doesn't even show volume/amplifier gain values).
For context : I'm trying to debug an audio issue with my laptop, where headphones output white noise when resuming from standby. The white noise doesn't change when increasing volume, but disappears only when powering down the codec.
This leads me to believe that the issue is likely caused by either a buggy ACPI or a change in the codec's configuration, or even both. My goal is to get as much data as I can on before/after states and compare them, but both methods described above failed for me.
I found that hdajackretask as part of the alsa-tools-gui package on my Debian Stretch GNU/Linux Os looks very interesting in getting the internal parts of the sound-card in the right configuration for what I wanted.
I'm just looking to see whether it is portable to my other OS as I have only seem to have stereo audio on FreeBSD and I think this would help. In your case it might help your to decode how things are configured and to spot any changes. The normal screen is at least helpful in determining which connection is which:
What looks more interesting is the "Advanced override":
I am about to reboot and see whether I have managed to turn the grey connector be the "side"-channel outputs....
Is it possible to record audio to a variable/RAM in LiveCode?
Normal recording requires to use a file, but I'm trying to figure out a way to not have to use the extra step of writing to disk, only to then read it from disk and send through sockets.
This is currently impossible and there is no good way to stream content from within LiveCode. When I tried to use video recording and sockets at the same time, I ran into a bug that caused LiveCode (Revolution at the time) to crash. Looking into the crash files, it appeared to me that using the recording routines and the socket routines at the same time caused a memory address conflict. After sending roughly 1000 recorded frames through a socket, Revolution would inevitably crash. To my best knowledge, this problem has never been fixed.
I would recommend dedicated software for streaming. A possibility might be VLC. You can use VLC from the command line, which means that you can set up a stream from within LiveCode, using the shell() function.
How can I increase opencv video FPS in Linux on Intel atom? The video seems lagging when processing with opencv libraries.
Furthermore, i m trying to execute a program/file with opencv
system(/home/file/image.jpg);
however, it shows Access Denied.
There are several things you can do to improve performance. Using OpenGL, GPUs, and even just disabling certain functions within OpenCV. When you capture video you can also change the FPS default which is sometimes set low. If you are getting access denied on that file I would check the permissions, but without setting the full error it is hard to figure out.
First is an example of disabling conversion and the second is setting the desired FPS. I think these defines are changed in OpenCV 3 though.
cap.set(CV_CAP_PROP_CONVERT_RGB , false);
cap.set(CV_CAP_PROP_FPS , 60);
From your question, it seems you have a problem that your frame buffer is collecting a lot of frames which you are not able to clear out before reaching to the real-time frame. i.e. a frame capture now, is processed several seconds later. Am I correct in understanding?
In this case, I'd suggest couple of things,
Use a separate thread to grab the frames from VideoCapture and then push these frames into a queue of a limited size. Of course this will lead to missing frames, but if you are interested in real time processing then this cost is often justified.
If you are using OOP, then I may suggest using a separate thread for each object, as this significantly speeds up the processing. You can see several fold increase depending on the application and functions used.
I'm looking into making some software that makes the keyboard function like a piano (e.g., the user presses the 'W' key and the speakers play a D note). I'll probably be using OpenAL. I understand the basics of digital audio, but playing real-time audio in response to key presses poses some problems I'm having trouble solving.
Here is the problem: Let's say I have 10 audio buffers, and each buffer holds one second of audio data. If I have to fill buffers before they are played through the speakers, then I would would be filling buffers one or two seconds before they are played. That means that whenever the user tries to play a note, there will be a one or two second delay between pressing the key and the note being played.
How do you get around this problem? Do you just make the buffers as small as possible, and fill them as late as possible? Is there some trick that I am missing?
Most software synthesizers don't use multiple buffers at all.
They just use one single, small ringbuffer that is constantly played.
A high priority thread will as often as possible check the current play-position and fill the free part (e.g. the part that has been played since the last time your thread was running) of the ringbuffer with sound data.
This will give you a constant latency that is only bound by the size of your ring-buffer and the output latency of your soundcard (usually not that much).
You can lower your latency even further:
In case of a new note to be played (e.g. the user has just pressed a key) you check the current play position within the ring-buffer, add some samples for safety, and then re-render the sound data with the new sound-settings applied.
This becomes tricky if you have time-based effects running (delay lines, reverb and so on), but it's doable. Just keep track of the last 10 states of your time based effects every millisecond or so. That'll make it possible to get back 10 milliseconds in time.
With the WinAPI, you can only get so far in terms of latency. Usually you can't get below 40-50ms which is quite nasty. The solution is to implement ASIO support in your app, and make the user run something like Asio4All in the background. This brings the latency down to 5ms but at a cost: other apps can't play sound at the same time.
I know this because I'm a FL Studio user.
The solution is small buffers, filled frequently by a real-time thread. How small you make the buffers (or how full you let the buffer become with a ring-buffer) is constrained by scheduling latency of your operating system. You'll probably find 10ms to be acceptable.
There are some nasty gotchas in here for the uninitiated - particularly with regards to software architecture and thread-safety.
You could try having a look at Juce - which is a cross-platform framework for writing audio software, and in particular - audio plugins such as SoftSynths and effects. It includes software for both sample plug-ins and hosts. It is in the host that issues with threading are mostly dealt with.
Here's what I want to do:
I want to allow the user to give my program some sound data (through a mic input), then hold it for 250ms, then output it back out through the speakers.
I have done this already using Java Sound API. The problem is that it's sorta slow. It takes a minimum of about 1-2 seconds from the time the sound is made to the time the sound is heard again from the speakers, and I haven't even tried to implement delay logic yet. Theoretically there should be no delay, but there is. I understand that you have to wait for the sound card to fill up its buffer or whatever, and the sample size and sampling rate have something to do with this.
My question is this: Should I continue down the Java path trying to do this? I want to get the delay down to like 100ms if possible. Does anyone have experience using the ASIO driver with Java? Supposedly it's faster..
Also, I'm a .NET guy. Does this make sense to do with .NET instead? What about C++? I'm looking for the right technology to use here, and maybe a good example of how to read/write to audio input/output streams using your suggested technology platform. Thanks for your help!
I've used JavaSound in the past and found it wonderfully flaky (and it keeps changing between VM releases). If you like C#, use it, just use the DirectX APIs. Here's an example of doing kind of what you want to do using DirectSound and C#. You could use the Effects plugins to perform your 250 ms echo.
http://blogs.microsoft.co.il/blogs/tamir/archive/2008/12/25/capturing-and-streaming-sound-by-using-directsound-with-c.aspx
You may want to look into JACK, an audio API designed for low-latency sound processing. Additionally, Google turns up this nifty presentation [PDF] about using JACK with Java.
Theoretically there should be no delay, but there is.
Well, it's impossible to have zero delay. The best you can hope for is an unnoticeable delay (in terms of human perception). It might help if you describe your basic algorithm for reading & writing the sound data, so people can identify possible problems.
A potential issue with using a garbage-collected language like Java is that the GC will periodically run, interrupting your processing for some arbitrary amount of time. However, I'd be surprised if it's >100ms in normal usage. If GC is a problem, most JVMs provide alternate collection algorithms you can try.
If you choose to go down the C/C++ path, I highly recommend using PortAudio ( http://portaudio.com/ ). It works with almost everything on multiple platforms and it gives you low-level control of the sound drivers without actually having to deal with the various sound driver technology that is around.
I've used PortAudio on multiple projects, and it is a real joy to use. And the license is permissive.
If low latency is your goal, you can't beat C.
libsoundio is a low-level C library for real-time audio input and output. It even comes with an example program that does exactly what you want - piping the microphone input to the speakers output.
It's possible with JavaSound to get end-to-end latency in the ballpark of 100-150ms.
The primary cause of latency is the buffer sizes of the capture and playback lines. The bufferSize is set when opening the lines:
capture: TargetDataLine#open(AudioFormat format, int bufferSize)
playback: SourceDataLine#open(AudioFormat format, int bufferSize)
If the buffer is too big it will cause excess latency, but if it's too small it will cause stuttery playback. So you need to find a balance for your applications needs and your computing power.
The default buffer size can be checked with DataLine#getBufferSize when calling #open(AudioFormat format). The default size will vary based on the AudioFormat and seems to be geared for high latency, stutter free playback applications (e.g. internet streaming). If you're developing a low latency application, the default buffer size is much too large and should be changed.
In my testing with a 16-bit PCM AudioFormat, a buffer size of 1024 bytes has been pretty close to ideal for low latency.
The second and often overlooked cause of audio latency is any other activity being done in the capture or playback threads. For example, logging messages to console can introduce 10's of ms of latency. Turn it off.