I'm trying to process a video using tensorflow in node.js (i.e. on the server - I don't have a web page). I need to process each frame in the video individually. I see some people are using ffmpeg to generate individual image files from the video but that seems wasteful as it creates files on the filesystem. I would prefer to grab each frame as a base64 string in memory. I've got this working using OpenCV4Node but am wondering if there are any lighter weight solutions. Is anyone already doing this? Any help would be appreciated :-)
Related
I want to read audio frequency data from video mp4 file get it as array (not mp3 file) that’s it, no need to do anything fancy
Currently I ‘m doing it it with webaudio api in javascript
However i need to do it with nodejs
i want to make this as fast as possible
I don’t care about the video frames data Or anything else
I’m trying with ffmpeg it seems very hard
if there is another way with fs maybe ?!!
Thank you in advance
I need to make a video of an audio equalizer.
So i need a script that analyses audio every frame, and extracts the frequency apectrum so i can draw that somehow and make an equalizer.
The first part of the problem is easily solvable on frontend as there is a myriad of open source equalizer visualisations in canvas.
The thing works nicely in browser but i have a problem to make an mp4 of that.
Ive tried using headless browsers(pupeteer and phantomjs) to capture frames from canvas, but i could not get the framerate above 10fps, resulting in unacceptable video quality and sync issues when connecting the jpg frames and mp3 via ffmpeg. The plan was to speed it up, so you dont have to wait for the full audio length to finish to get an mp4, but i cant even get it to show above 10fps on regular playback speed.
I feel the tech i thought would work is not there yet, and i might be in need of a different approach.
The only condition is that it has to run as a script on a linux server. So any programmimg language or any equalizer design will work.
Any ideas or resources are more than welcome. Thanks
I am trying to encode 2 videos side by side, sync'd by the audio of the 2 clips. I can successfully encode the 2 videos side by side and select the audio from one of the input streams. However the system we are using to record the 2 videos does not start and stop the recording at the same time (could be up to a second different between cameras). Basically we are using a CCTV system to capture what's going on in a room from multiple angles. We export the 2 cameras between 2 timestamps and due to the way the system records the videos the start of the 2 clips are not the same point in time.
e.g. Export videos between 09:00:00:000 and 09:10:00:000
Video 1 - exports from 08:59:59:123 to 09:10:00:123
Video 2 - exports from 08:59:59:789 to 09:10:00:789
Therefore when video 1 and video 2 are stitched together side by side, they are out of sync by 666ms (which is very noticeable in the encoded video)
Both input streams have (near) identical audio and are both in the exact same format. We are currently placing these videos into Premiere Pro and syncing these videos by the audio and exporting them side by side, however we have a project where we need to do a lot of these in quick succession and this is not really an option. We need to look at scripting this.
Does anyone know if FFMPEG can do this? Or anything else?
Any info would be greatly appreciated.
You can use audio-offset-finder in bash file to calculate offset, cut of the head from one of the video, stitch them together ( like stated here ).
You would need to extract audio streams into separate files and use finder to calculate offset.
offset=`audio-offset-finder --find-offset-of file1.wav --within file2.wav`
I want to be able to set the "Title" and "Comments" (listed in properties->details) of some mp3 files in Windows using python. Is this possible, perhaps with a library like PyWin32? Also, would these details be visible in other operating systems or are they Windows-specific? Thanks.
Simple Answer:
Yes, you can set 'Title' and 'Comments' (and many other fields) of an mp3 file in Windows using Python.
Also, the details are visible on all operating systems and are not windows specific.
First you have to understand what is mp3 file and how data is organized within an mp3 file.
Detailed Answer:
Raw audio consumes a lot of size. For example, an audio signal of 10 sec sampled 48 kHz and having a bit depth of 16 bits per sample will be of size 10*48000*16 bits, which is close to 1 MB. So, for a 5 minute song, it will almost take 30 MB. But, if you observe, most 5 min mp3 songs are of size around 5 MB (of course it depends on sampling frequency, bit depth and amount of compression used). How is it possible? It is possible because we compress the data using signal processing techniques which in itself is a big topic altogether which we will not discuss here. So, to create an mp3 file we need something called encoder which converts the raw audio data to compressed data and every time you play an mp3 song, decoder is used which converts the data from compressed format to raw audio, which is what you can only listen. So, compression is done for saving storage and also transmission bandwidth (basically saving amount of data to be transmitted over internet).
Now, coming to how data is organized inside an mp3 file. mp3 file will obviously contain the compressed data. In addition many mp3 files contain some meta data (like Title and Comments you mentioned in your question). There are several formats for storing this meta data. So, a decoder which is decoding mp3 file should also support decoding of meta-data, then only you can see the information, other wise you can't see. The meta data is operating system independent, and can be seen on any operating system provided you have a proper decoder.
Finally, yes you can edit the meta data on windows (for that matter on any OS) using python. If you want to do this, using only python without any library, you need to understand how data is organized inside an mp3 file, find the meta-data inside it, edit it and store it back. But, there are libraries and packages in python which support editing meta-data of mp3 file. You can use them directly. Also, the meta data is independent of OS, and once you edit your properties, you should be able to see the properties in any OS provided the decoder you use has the support.
Some links which will help you:
mp3 tag tool
Another stack overflow question which gives details about libraries that support viewing and editing of meta data using Python
I'm wondering if it's possible to draw an audio channel of a video or audio file as an image using ffmpeg, or if there's another tool that would do it on Win2k8 x64. I'm doing this as part of an encoding process after a user uploads a video or audio file.
I'm using ColdFusion 10 to handle the upload and calling cfexecute to run ffmpeg.
I need the image to look something like this (without the horizontal lines):
You can do this programmatically very easily.
Study the basics of FFmpeg. I suggest you to compile this sample. It explains how to open a video/audio, identify the streams and loop over the packets.
Once you have the data packet (in this case you are interested only in the audio packets). You will decode it (line 87 of this document) and obtain the raw data of an audio. It's the waveform itself (the analogue "bitmap" for an audio).
You could also study this sample. This second example is how to write a video/audio file. You don't want to write any video, but with this sample you can easily understand how the audio raw data packet works, if you see the functions get_audio_frame() and write_audio_frame().
You need to have some knowledge about creating a bitmap. Any platform has an easy way to do that.
So, the answer for you: YES, IT IS POSSIBLE TO DO THIS WITH FFMPEG! But you have to code a little bit in order to get what you want...
UPDATE:
Sorry, there are ALSO built-in features for this:
You could use those filters... or
showspectrum, showwaves, avectorscope
Here are some examples on how to use it: FFmpeg Filters - 12.22 showwaves.