Do Anyone know realtime encode video in node with ffmpeg ? I know transloadit was done well on this. Any idea ?
https://transloadit.com/blog/2010/12/realtime-encoding-over-150x-faster
Co-founder at Transloadit here : ) We used pipes. Node.js allows us to see the data as it is still being uploaded (we used our node-formidable module). FFmpeg allows using stdin for input via ffmpeg -i -. So you can pipe uploaded bytes into that spawned child_process's stdin, and that's that : )
Off-topic, we later deprecated the feature. It turned out the market for it was less interested than we had imagined, and it sadly introduced enough operational headaches that we said goodbye to this, by us, beloved feature.
Use Fluent FFMpeg. A tremendous module that can transcode on the fly:
https://github.com/schaermu/node-fluent-ffmpeg
Related
As much as it matters my scenario is developing an accessibility application not any kind of malicious eavesdropping, whereas also within this scenario there are various research and development implied scenarios, all of which should greatly benefit from being able to read the microphone audio stream by multiple simultaneously running unrelated processes such as recording tools and/or different versions of my own code.
Problem Statement
I am reading a microphone input stream using a high level python API like follows:
import sounddevice
audio_stream = sounddevice.InputStream(
device=self.microphone_device,
channels=max(self.channels),
samplerate=self.audio_props['sample_rate'],
blocksize=int(self.audio_props['frame_elements_size']),
callback=self.audio_callback)
I would like to learn whether it is possible (on linux) to read the microphone audio stream simultaneously to another program such as Google Meet / Zoom reading it. I.e. effectively share the audio stream.
As is with the mentioned python wrapper, it is no big surprise that when the above code is started while a video call is in progress, it will simply fail to open the stream:
Expression 'paInvalidSampleRate' failed in
'src/hostapi/alsa/pa_linux_alsa.c', line: 2043
Expression 'PaAlsaStreamComponent_InitialConfigure( &self->playback, outParams, self->primeBuffers, hwParamsPlayback, &realSr )'
failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 2716
Expression 'PaAlsaStream_Configure( stream, inputParameters, outputParameters, sampleRate, framesPerBuffer, &inputLatency, &outputLatency, &hostBufferSizeMode )'
failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 2837
Admittedly, I am not very well versed with ALSA terminology and in general the sound stack on linux yet.
My question is, can this be accomplished directly using ALSA library API, or otherwise via other sound stacks or sound system configuration? Or if all else is not meant to work, via a proxy program/driver that is able to expose an audio buffer to multiple consumers without incurring noticeable degradation in audio stream latency?
You can do this directly with ALSA. Dsnoop should do the trick. It is a plugin included with ALSA that allows sharing input streams.
From the page I linked above:
dsnoop is the equivalent of the dmix plugin, but for recording sound. The dsnoop plugin allows several applications to record from the same device simultaneously.
From the ALSA docs:
If you want to use multiple input(capture) clients you need to use the dsnoop plugin:
You can poke around there for details on how to use it. This issue on GitHub will also help you get started, it details how to configure the dsnoop interface so you can read from it with pyaudio.
Update
To configure ALSA, edit /etc/asound.conf with something like this (from the ALSA docs on dsnoop):
pcm.mixin {
type dsnoop
ipc_key 5978293 # must be unique for all dmix plugins!!!!
ipc_key_add_uid yes
slave {
pcm "hw:1,0"
channels 2
period_size 1024
buffer_size 4096
rate 44100
periods 0
period_time 0
}
bindings {
1 1
1 0
}
}
You can test to see if your configuration works with something like this:
arecord -d 30 -f cd -t wav -D pcm.mixin test.wav
So, this is more an audio question than a python question I guess. :)
Depending on the API, Streams can be device exclusive or not. ASIO for professional audio for example is often device exclusive, so just one application(like a DAW) has access to it. On Windows for example you can turn this on and off as seen here:
https://help.ableton.com/hc/en-us/articles/209770485-Disabling-exclusive-mode-for-ASIO-interfaces
Most Python packages like pyaudio and so on are just providing bindings for portaudio, which does the heavy lifting, so also have a look at the portaudio documentation. Portaudio "combines" all the different APIs like ASIO,ALSA,WASAPI,Core Audio, and so on.
For ALSA to create more than one Stream at the same time you might need dmix, have a look at this Stackoverflow question:
https://unix.stackexchange.com/questions/355662/alsa-doesnt-work-when-multiple-applications-are-opened
I have trouble using ffprobe from node.js. I need the audio lengths MP3 files. There is an npm package, get-audio-duration for this.
The package calls ffprobe through an execa command. It works well for .flac files both when when using a filename and a stream. However for .mp3 files it fails for streams.
I suspected some problems with execa so I checked from the command line (on Windows 10):
type file.mp3 | ffprobe -
(Where I left out the parameters to ffprobe for clarity.)
This kind of works, but says duration=N/A.
It looks to me like ffprobe didn't get the info that the input is finished. Or, it dint care about it. (There is a 4 year old bug report about this on the ffmpeg issue site which was closed for no obvious reason.)
Is it possible to somehow tell ffprobe that the pipe has ended?
It's not a matter of noticing that the pipe has ended.
ffprobe uses a different way of determining the file size than is allowed by piping stdout to stdin
See https://trac.ffmpeg.org/ticket/4358
Maddening gap in an app I'm developing is there appears to be little (or no) support for AVI in the HTML5 video implementation. So, I need a workaround that is cross-platform, and package-able with my electron app.
Videos are hosted locally
I'm not averse to encoding on the fly (ffmpeg avi -> mp4 and use HTML5 natively?)
WebChimera appears dying due to VLC and Electron changes (devs can't keep up) (Is there another npm package that can do this?)
A wrapper that calls a native VLC instance might work -- but how do I ensure that VLC is available on the system with my packaging?
Should I just spawn a native app in a separate window (ie, Totem on Linux)? (seems clunky)
Latest videoj-java plugin apparently has issue (https://github.com/Afterster/videojs-java/issues/2) and adding another layer (java) to the electron stack seems somehow unsavory.
FFBinaries (https://github.com/vot/ffbinaries-node) seems promising... but oddly FFPlay is not available for Linux (though I suspect my linux consumers likely have a ffmpeg already installed).
NB: Files are decidedly AVI. I can't change this.
Any hints / pointers greatly appreciated!
UPDATE
On my system, using ffmpeg to convert:
ffmpeg -i infile.AVI -vcodec copy -acodec copy outfile.mp4
Takes no time at all (they are short videos):
real 0m0.138s
user 0m0.100s
sys 0m0.032s
So I'm leaning toward packaging ffmpeg with my program and converting before loading.
Take a look at this project:
https://github.com/RIAEvangelist/electron-video-player
According to the known supported formats:
https://github.com/RIAEvangelist/electron-video-player#known-supported-video-types
it supports:
mp4
webm
ogg
mov (MPEG4 | H.264)
avi (MPEG4 | H.264)
mkv (MPEG4 | H.264)
m4v (MPEG4 | H.264)
Take a look at its source code and see if you can implement it similarly.
You said that you need AVI support but AVI is just a container - if you need other codecs than the ones supported by this project then you will still need to transcode it first.
If you cannot do it like this then you may try using something similar to:
https://www.npmjs.com/package/mplayermanager
and bundle mplayer with your app, or some other player.
According to this SO answer, Electron now suports multiple video formats in the <video> tag, including .mkv, .avi and other formats. You don't need to rely on an external player.
I would like to use MPEG-DASH technology in situations where I am constantly receiving a live video stream from a client. The Web server gets a live video stream, keeps generating the m4s file, and declares it in mpd. So the new segment can be played back constantly.
(I'm using FFMPEG's ffserver. So the video stream continues to accumulate in /tmp/feed1.ffm file.)
Using MP4Box seems to be able to generate mpd, init.mp4, m4s for already existing files. But it does not seem to support live streaming.
I want fragmented mp4 in segment format rather than mpeg-ts.
A lot of advice is needed!
GPAC maintainer here. The dashcast project (and likely its dashcastx replacement from our Signals platform should help you). Please open issues on github if you have any issues.
Please note that there are some projects like this one using FFmpeg to generate some HLS and then GPAC to ingest the TS segments to produce MPEG-DASH. This introduces some latency but proved to be very robust.
Below information may be useful.
latest ffmpeg supports the live streaming and also mp4 fragmenting.
Example command
ffmpeg -re -y -i <input> -c copy -f dash -window_size 10 -use_template 1 -use_timeline 1 <ClearLive>.mpd
I have a windows phone 8 app which plays audio streams from a remote location or local files using the BackgroundAudioPlayer. I now want to be able to add audio effects, for example, reverb or echo, etc...
Please could you advise me on how to do this? I haven't been able to find a way of hooking extra audio processing code into the pipeline of audio processing even through I've read much about WASAPI, XAudio2 and looked at many code examples.
Note that the app is written in C# but, from my previous experience with writing audio processing code, I know that I should be writing the audio code in native C++. Roughly speaking, I need to find a point at which there is an audio buffer containing raw PCM data which I can use as an input for my audio processing code which will then write either back to the same buffer or to another buffer which is read by the next stage of audio processing. There need to be ways of synchronizing what happens in my code with the rest of the phone's audio processing mechanisms and, of course, the process needs to be very fast so as not to cause audio glitches. Or something like that; I'm used to how VST works, not how such things might work in the Windows Phone world.
Looking forward to seeing what you suggest...
Kind regards,
Matt Daley
I need to find a point at which there is an audio buffer containing
raw PCM data
AFAIK there's no such point. This MSDN page hints that audio/video decoding is performed not by the OS, but by the Qualcomm chip itself.
You can use something like Mp3Sharp for decoding. This way the mp3 will be decoded on the CPU by your managed code, you can interfere / process however you like, then feed the PCM into the media stream source. Main downside - battery life: the hardware-provided codecs should be much more power-efficient.