im trying to create a visualizer with Gstreamer 1.0.
When I get the raw audio data from the pipeline in the format of a GstBuffer,
and then the Buffer is mapped with gst_buffer_map,
the corresponding map.data (link)
is of format guint8, so it is between 0 and 255. The map size is slightly above 4000.
How can I interpret these values? How are the audio frequencies contained within them?
I would be very thankful for a helpful answer.
regards,
tagelicht
You're probably using an appsink and calling gst_app_sink_set_caps() on it during the creation to set up the format that will be received on it.
There is also gst_app_sink_get_caps().
Related
I'm wondering if it's possible to draw an audio channel of a video or audio file as an image using ffmpeg, or if there's another tool that would do it on Win2k8 x64. I'm doing this as part of an encoding process after a user uploads a video or audio file.
I'm using ColdFusion 10 to handle the upload and calling cfexecute to run ffmpeg.
I need the image to look something like this (without the horizontal lines):
You can do this programmatically very easily.
Study the basics of FFmpeg. I suggest you to compile this sample. It explains how to open a video/audio, identify the streams and loop over the packets.
Once you have the data packet (in this case you are interested only in the audio packets). You will decode it (line 87 of this document) and obtain the raw data of an audio. It's the waveform itself (the analogue "bitmap" for an audio).
You could also study this sample. This second example is how to write a video/audio file. You don't want to write any video, but with this sample you can easily understand how the audio raw data packet works, if you see the functions get_audio_frame() and write_audio_frame().
You need to have some knowledge about creating a bitmap. Any platform has an easy way to do that.
So, the answer for you: YES, IT IS POSSIBLE TO DO THIS WITH FFMPEG! But you have to code a little bit in order to get what you want...
UPDATE:
Sorry, there are ALSO built-in features for this:
You could use those filters... or
showspectrum, showwaves, avectorscope
Here are some examples on how to use it: FFmpeg Filters - 12.22 showwaves.
i am wondering if there is any way to cycle through a .wav file to get the amplitude/DB of a specific point in the wav file. i am reading it into a byte array now but that has no help to me from what i can see.
i am using this in conjunction with some hardware i have developed that encodes light data into binary and outputs audio. i wont get into the details but i need to be able to do this in c# or c++. i cant find any info on this anywhere. i have never programmed anything relating to audio so excuse me if this is a very easy thing.
i dont have anything started since this is the starting point so if anybody can point me to some functions, libraries, or methods to being able to collect the amplitude of the wave at a specific time in the file, i would greatly appreciate it.
i hope this is enough info, and thank you in advance if you are kind enough to help.
It is possible and it is done in a straightforward way: the file with PCM audio contains one value for every channel, for every (1/sample-rate) of second.
The values however might vary: 8-bit, 16-bit, single precision floating point values. You certainly have to take this into account and this is the reason you cannot take the bytes from byte array directly.
The .WAV file also has a header preceding the actual payload.
I'm working with C# and a usb webcam that supports YUY2 or MJPG image formats. Thus far I've always had it in YUY2 mode and that works fine. Recently I tried changing the format to MJPG thinking that it would then feed my program one JPEG image per frame capture. It appears to almost do that. When I try to display the buffer, my app always takes an exception which is vague, but seems to indicate that the stream is invalid. I then copied one of the buffers to a file and tried to view it with IrfanView and it tells me that there is no huffman table. Looking at the buffer with a binary editor, I see it does have the SOI and EOF JPEG markers (and several others); however, it doesn't contain a huffman table marker. Any ideas what I'm doing wrong here? I've read a bit about JPEG and apparently there are cases where images can use a standard huffman table to reduce file size; however, if that's the case, how do I insert this into the image (if appropriate)?
This is with reference to Microsoft Lifecam by the way.
Part of the Motion-JPEG standard for AVI files is that a fixed Huffman table will be used so that it doesn't have to be stored in every frame.
I am wondering on how to estimate where I am currently in an audio with regards to time, by using the data.
For example, I read data by byte[8192] blocks. How can I know how much byte[8192] is equivalent to in time?
If this is some sort of raw-ish encoding, like PCM, this is simple. The length in time is a function of the sample rate, bit depth, and number of channels. 30 seconds of 16-bit audio at 44.1kHz in mono is 2.5MB. However, you also need to factor in headers and container format crapola. WAV files for example can have a lot of other stuff in them.
Compressed formats are much more tricky. You can never be sure where you are without playing through the file to get to where you are. Of course you can always guesstimate based on the percentage of the file length, if that is good enough for your case.
I think this is not what he was asking.
First you have to tell us what kind of data you are using. WAV? MP3? Usually without knowing where that block came from - so you know if you have some kind of frame information and where to find it - you are not able to determine that block's position.
If you have the full stream and this data then you can do a search
Does all audio format has a header for audio length (in second)?
If not, what kind of audio format has that information embedded in the header.
Thank you.
Not necessarily. Typical wav files will have a wave format chunk (WAVEFORMATEX if you're coding on Windows) which contains the sample rate and number of bits per sample. Most of the WAV files you'll tend to come across are in PCM format where you know that there is always the same number of samples per second and bits per sample, so from the size of the file and these values you can work out the duration exactly.
There are other types of WAV file though which may be compressed (though these are much rarer) and for those you'll need to use the 'average bytes/sec' field of the WAVE header to work out the length.
If you're using AIFF (largely used on macs) then this has similar data members in the header.
Getting the length from an MP3 file is more difficult -- some suggestions are in this other question