Safely parsing the output of file (or libmagic) - linux

I'm writing some code where I rely on the file utility to determine the file type of arbitrary files, typically audio files. For the most part, it works great, an ogg file for example might give the following output:
Ogg data, Vorbis audio, mono, 44100 Hz, ~80000 bps, created by: Xiph.Org libVorbis I (1.0.1)
A simple regexp can classify this as ogg vorbis.
But for some other file types, file tries to get clever, an nsf (NES sound format) file for example, can yield this output:
NES Sound File ("The Legend of Zelda" by Konchano, copyright 1987 Nintendo), version 1, 8 tracks, NTSC
"NES Sound File" is clear enough, but it is followed by a string of unstructured data that is clearly just copied from the file itself. A malicious user could create an nsf file where this string is replaced by something like "Ogg data, Vorbis audio", making classification a lot harder.
Now let's say I fix this by discarding anything within parentheses (ignoring the fact that the title of the track could itself contain parentheses), along comes a Protracker module:
4-channel Protracker module sound data Title: "space_debris"
Again, untrusted data straight from the file, in a different position, now with the prefix "Title:". I can attempt to filter it out but really this is becoming a hassle.
I'm not finding any help in the man page. Is there really no way to tell file not to mix these unsafe strings into its output? Or is file simply not the right tool for this job?

Related

File information of .raw audio files using terminal in linux

How to get file information like sampling rate, bit rate etc of .raw audio files using terminal in linux? Soxi works for .wav files but it isn't working for .raw.
If your life depended on discovering an answer you could make some assumption to tease apart the unknowns ... however there is no automated way since the missing header would give you the easy answers ...
The audio analysis tool called audacity allows you to open up a RAW file, make some guesses and play the track
http://www.audacityteam.org
In audacity goto File -> Import -> Raw Data...
Above settings are typical for audio ripped from a CD ... toy with trying stereo vs mono for starters.
Those picklist widgets give you wiggle room to discover the format of your PCM audio given that the source audio is something when properly rendered is recognizable ... would be harder if the actual audio was noise
However if you need a programmatic method then rolling your own solution to ask those same questions which appear in above window is possible ... is that what you need or will audacity work for you ? We can go down the road of writing code to play off the unknowns mentioned in #Frank Lauterwald's comment
To kick start discovering this information programmatically, if the binary raw audio is 16 bit then each audio sample (point on the audio curve) will consume two bytes of your PCM file. For mono audio then the following two bytes would be your next sample, however if its stereo then these two following bytes would be the sample from the other channel. If more than two channels then just repeat. Typical audio is little endian. Sampling rate is important when rendering the audio, not when programmatically parsing raw bytes. One approach would be to create an output file with a WAV header followed by your source PCM data. Populate the header with answers from your guesswork. This way you could listen to this output file to help confirm your guesses.
Here is a sample 500k mono PCM audio file signed 16 bit which can be imported into audacity or used as input to rolling your own identification code
The_Constructus_Corporation_Long_Street-ycexQvMy03k_excerpt_mono.pcm

Merging of two AAC files into a single file

I am trying to merge two different AAC audio files and a H264 video file to form a single TS file using C++ code. I have been successful in it. So now my TS file possess the following order. First, video part from the video file, then audio part from the first audio file and then audio part from the second audio file and then again the video part and it goes on the same way. On hearing the resulting file, I recognized the presence of the different audio files with the video.The problem is that the resulting audio ain't that much cleared. Distortions can be recognized making it unclear to hear. Also note that the resulting audio seems slow as compared to the original.Can anyone guide me in getting off those distortions and procuring the exact replica of my original files ?
Thanks,
Ashish.

Creating preview audio clips from m4a files

Another post here answered the question of creating 30 second preview clips from WAV audio files (Create mp3 previews from wav and aiff files). My needs slightly overlap, but differing details are beyond my knowledge.
Requirements/Options: clip length; beginning & ending fade length; input filetypes: m4a/AAC/AIFF; output filetype: mp3; kbps (e.g. 192); original files unaltered; suffix new mp3 names with " (Preview)"
Limitations: no uploading of original files to a server (desktop processing); no compiling (unix/Terminal/Bash script only); recursive processing of files in sub-directories
Any/All assistance and advice is welcome.
You'll most likely get the best results with a DAW (digital audio workstation) and an audio file converter.
For a DAW, Reaper comes with a 60 day trial, and it has everything you need to cut the songs where you need and to do fade ins/fade outs, and other effects if you'd like.
www.reaper.fm
Simply use a converter to convert the m4a file to .wav, .mp3 or whatever you prefer, and then if you need it back in m4a, convert it back. I say this because some DAWs can't work with m4a files, but if which ever one you choose to work with can then no conversion is necessary,
There are many options for what DAW and what converter you use, I recommend Reaper for a DAW, and most converters essentially do the same thing, so it doesn't make much of a difference which one you choose.
Hope this helps!

Difference between audio encoding/decoding and format conversion

Recently i have been trying to convert an audio file from one format to another through ffmpeg. i was trying to do some google but results made me a little confused about the difference between encoding and decoding an audio file and converting from one format to another.
Let me describe it this way: There are several different file formats for video files (sometimes also called "wrappers"). There are also several different codecs which can be used to encode (or compress) the audio and video. Audio and video use different codecs - and the encoded formats can be sorted in different file types/formats.
So when you talk about "encoding" vs. "converting" a couple of things come into play.
"Encoding" would be the act of taking audio/video and encoding them into a given codec(s). "Converting" implies having stuff in one format, but wanting it in another. There are two ways of looking at this:
Often called "repackaging" - this is when the video (for example) has been encoded correctly (let's say h264, with a bunch of parameters), but you want it in a different file-type - maybe it's an .AVI and you wanted it in an .MP4. This doesn't involve changing the actual video - just re-wraping the h264 stream in a new "wrapper", and is thus a fast operation.
Re-encoding. Let's say your audio was in a MP3 format, and you wanted it in an AAC format. This would require decoding the entire MP3 stream, and re-encoding it into AAC.
Obviously you can also do "1" and "2" together.
Refer Formats and Codecs for detailed information.
Hope it helps!

Hashing raw audio data

I'm looking for a solution to this task: I want to open any audio file (MP3,FLAC,WAV), then proceed it to the extracted form and hash this data. The thing is: I don't know how to get this extracted audio data. DirectX could do the job, right? And also, I suppose if I have fo example two MP3 files, both 320kbps and only ID3 tags differ and there's a garbage inside on of the files mixed with audio data (MP3 format allows garbage to be inside) and I extract both files, I should get the exactly same audio data, right? I'd only differ if one file is 128 and the other 320, for example. Okay so, the question is, is there a way to use DirectX to get this extracted audio data? I imagine it'd be some function returning byte array or something. Also, it would be handy to just extract whole file without playback. I want to process hundreds of files so 3-10min/s each (if files have to be played at natural speed for decoding) is way worse that one second for each file (only extracting)
I hope my question is understandable.
Thanks a lot for answers,
Aaron
Use http://sox.sourceforge.net/ (multiplatform). It's faster than realtime as you'd like, and it's designed for batch mode much more than DirectX. For example, sox -r 48k -b 16 -L -c 1 in.mp3 out.raw. Loop that over your hundreds of files using whatever scripting language you like (bash, python, .bat, ...).

Resources