How to minimize Liquidsoap's processing latency?
I have written a very simple liquidsoap script to test latency:
#!/usr/bin/env liquidsoap
set("log.stdout",true)
set("log.file",false)
stream = input.jack(clock_safe=false)
output.jack(stream)
When I connect the ports with music player (mpd) and soundcard, music playback starts and stops with about 0.5s latency. I have JACK buffer size set to 1024 so the latency should be about 0.02s. When I connect mpd directly to soundcard, the latency is unnoticable so it's definitely problem with Liquidsoap.
Is there any configuration option I'm missing which causes unwanted buffering when set to default value? I tried various clock_safe values for input and output but they do nothing.
According to the documentation, input.jack takes a buffer_size parameter:
buffer_size (int – defaults to 2): Set buffer size, in frames. Must be >= 1.
Looks like its set to 2 by default, so not sure if you are going to gain much by setting it to 1...
Related
as I said in the title, I need to record my screen from an electron app.
my needs are:
high quality (720p or 1080p)
minimum size
record audio + screen + mic
low impact on PC hardware while recording
no need for any wait after the recorder stopped
by minimum size I mean about 400MB on 720p and 700MB on 1080p for a 3 to 4 hours recording. we already could achieve this by bandicam and obs and it's possible
I already tried:
the simple MediaStreamRecorder API using RecordRTC.Js; produces huge file sizes, like 1GB per hour for 720p video.
compressing the output video using FFmpeg; it can take up to 1 hour for 3 hours recording
save every chunk with 'ondataavailable' event and right after, run FFmpeg and convert and reduce the size and append all the compressed files (also by FFmpeg); there are two problems. 1, because of different PTS but it can be fixed by tunning compress command args. 2, the main problem is the audio data headers are only available in the first chunk and this approach causes a video that only has audio for the first few seconds
recording the video with FFmpeg itself; the end-users need to change some things manually (Stereo Mix), the configs are too complex, it causes the whole PC to work slower while recording (like fps drop; even if I set -threads to 1), in some cases after recording is finished it needs many times to wrap it all up
searched through the internet to find applications that can be used from the command line; I couldn't find much, the famous applications like bandicam and obs have command line args but there are not many args to play with and I can't set many options which leads to other problems
I don't know what else I can do, please tell me if u know a way or simple tool that can be used through CLI to achieve this and guide me through this
I end up using the portable mode of high-level 3d-party applications like obs-studio and adding them to our final package. I also created a js file to control the application using CLI
this way I could pre-set my options (such as crf value, etc) and now our average output size for a 3:30 hour value with 1080p resolution is about 700MB which is impressive
I am working on RTSP live Streaming. I am getting live stream on my android App using exoplayer RTSP stream player. But latency of that streaming is about 3 seconds. As latency on vlc media player is 1 second. so how to reduce latency in exoplayer. Is there any way please tell me
What you're facing is buffering latency. VLC uses its own engine and buffering algorithms. However, if you wanna reduce buffering latency on ExoPlayer, you gotta get yourself familiarized with LoadControl. ExoPlayer uses a DefaultLoadControl in default instantiation. This LoadControl is a class that belongs to the ExoPlayer library, and it determines the values of time durations the player should spend in order to buffer the stream. If you wanna reduce the delay, you gotta reduce the LoadControl buffer values.
Here's a quick snippet of creating a SimpleExoPlayer instance with a custom load control :
val customLoadControl = DefaultLoadControl.Builder()
.setBufferDurationsMs(minBuffer, maxBuffer, playbackBuffer, playbackRebuffer)
.build()
Parameters in a nutshell : minBuffer is the minimum buffered video duration, maxBuffer is the maximum buffered video duration, playbackBuffer is the required buffered video duration in order to start playing , playbackRebuffer is the required buffered video duration in case it failed and it retries.
In your case, you should set your values really low, especially the playbackBuffer and minBuffer parameters. You should mess around with small values (they are in milliseconds). Going with a value of 1000 for both minBuffer and playbackBuffer
How to use the custom load control : After building the custom load control instance and storing it in a variable, you should pass it when you're building your SimpleExoPlayer :
myMediaPlayer = SimpleExoPlayer.Builder(this#MainActivity)
.setLoadControl(customLoadControl)
.build()
What to expect:
Using the default values is always recommended. If you mess with the values, the app may crash, the stream may be stuck, or the player may get very glitchy. So manipulate the values intelligently.
Here is the javadocs DefaultLoadControl Javadocs
If you don't know what buffering is, exoplayer (or any other player) may need to buffer (load the upcoming portion of the video/audio and store it in memory, rendered, way faster to access and reduces playback problems. Streamed media however needs buffering because it comes in form of chunks. So each chunk that arrives, will eventually be buffered. If you set the required buffered duration to 1000, you are telling ExoPlayer that the first chunk of stream that arrives whose length is 1000 millisecond should be buffered and played right away. I believe there is no simpler way to explain this. Best of luck.
Currently, I use sox like this:
sox -d -e u-law --endian little -b 8 -c 1 -r 8000 -t ul - silence 1 0.3 1% 1 0.3 1%
For reference, this is recording audio from the default microphone and outputting little endian, ulaw formatted audio at 8 bits and a 8k rate. The effects filter trims audio until the noise hits a threshold for 0.3 seconds, then continues to record until there is 0.3 seconds of silence. All of this streams to stdout which I use to stream to a remote server.
I am using all of this to record a bit of voice and finish when I am done speaking. To trigger sox, I use specialized hardware to trigger the start of the recording. I can switch to using almost any audio format or codec as long as it supports on the fly formatting/encoding. My target platform is raspbian on the raspberry pi 2 B.
My ideal solution would be to use vad to stop the recording when the user is finished speaking. My hope is that this would work even with background chatter. However, the sox documentation on the vad effect states this:
The use of the norm effect is recommended, but remember that neither
reverse nor norm is suitable for use with streamed audio.
I haven't been able to piece parameters together to get vad and streaming working. Is it possible to use the vad effect to stop the recording of audio while still maintaining the stdin->sox->stdout piping? Are there better alternatives?
Is it possible to use the vad effect to stop the recording of audio while still maintaining the stdin->sox->stdout piping?
No. The vad effect can trim silence only from the front of the audio. So you could only use it to detect recording start, and not ending and pauses.
The reverse and norm filters need all the input data before they produce any data on output, that is why they cannot be used with streaming.
The key is to select a good threshold for silence filter so it takes "background chatter" as silence.
You could use also noisered (with a profile based on previous recordings) before silence to reduce noise triggering the recording, but this will also affect output and probably will not take "background chatter" as noise.
I want to capture audio on Linux with low latency in a program I'm writing.
I've run some experiments using the ALSA API, using snd_pcm_readi() to
capture sound, then immediately using snd_pcm_writei() to play it back.
I've tried playing with the number of frames captured, and the buffer size,
but I don't seem to be able to get the latency down to less than a second
or so.
Am I better off using PulseAudio or JACK? Can those be used to play the
captured audio?
To reduce capture latency, reduce the period size of the capture device.
To reduce playback latency, reduce the buffer size of the playback device.
Jack can play the captured audio (just connect the input ports to the output ports), but you still have to configure its periods/buffers.
Also see Relation between period size of speaker and mic and Recording from ALSA - understanding memory mapping.
I've doing some work on low latency audio programming,
My experience is, first, your capture buffer should be small, like 10ms period buffer. (let's assuming you're using 512 frame buffer, and 48000 sample rate).
Then, you should config your Output device start_threshold to at least 2 * frame size ( 1 * frame size if your don't have much process of recorded data).
For record device, like CL. said, use a relative small period size is better, but not too small to avoid too much irq.
Also, you can change your process schedule to FIFO schedule.
Then, hopefully, you will get about 20ms total latency.
I believe you should at first ensure that you are running a Linux kernel which actually allows you to achieve low typical latency.
There are several kernel compile-time configuration options which you might look into:
CONFIG_HZ_1000
CONFIG_IRQ_FORCED_THREADING
CONFIG_PREEMPT
CONFIG_PREEMPT_RT_FULL (available only with RT patch)
Apart from that, there are more things you can do in order to optimize your audio latency in Linux. Some starting reference points can be found there:
http://wiki.linuxaudio.org/wiki/real_time_info
I wrote a little program which records voice from the microphone and sends it over network and plays it there. I'm using PyAudio for this task. It works almost fine but on both computers i get errors from ALSA that an underrun occurred. I googled a lot about it and now I know what an underrun even is. But I still don't know how to fix the problem. Most of the time the sound is just fine. But it sounds a little bit strange if underruns occur. Is there anything I should take care of in my code? It feels like I'm doing an simple error and I miss it.
My system: python: python3.3, OS: Linux Mint Debian Edition UP7, PyAudio v0.2.7
Have you considered syncing sound?
You didn't provide the code, so my guess is that you need to have a timer in separate thread, that will execute every CHUNK_SIZE/RATE milliseconds code that looks like this:
silence = chr(0)*self.chunk*self.channels*2
out_stream = ... # is the output stream opened in pyaudio
def play(data):
# if data has not arrived, play the silence
# yes, we will sacrifice a sound frame for output buffer consistency
if data == '':
data = silence
out_stream.write(data)
Assuming this code will execute regularly, this way we will always supply some audio data to output audio stream.
It's possible to prevent the underruns by filling silence in if needed.
That looks like that:
#...
data = s.recv(CHUNK * WIDTH) # Receive data from peer
stream.write(data) # Play sound
free = stream.get_write_available() # How much space is left in the buffer?
if free > CHUNK # Is there a lot of space in the buffer?
tofill = free - CHUNK
stream.write(SILENCE * tofill) # Fill it with silence
#...
The solution for me was to buffer the first 10 packets/frames of the recorded sound. Look at the snippet below
BUFFER = 10
while len(queue) < BUFFER:
continue
while running:
recorded_frame = queue.pop(0)
audio.write(recorded_frame)