I checked through the questions asked on SO on audio metadata, but could not find one which answers my doubt. Where exactly is the metadata of audio files stored, and in what form? Is it in the form of files or in a database? And where is this database of files stored?
Thank you Michelle. My basic confusion was whether the metadata is stored as a part of the file or in a separate file which is stored somewhere else in the file system - like inode in case of Unix like systems. ID3 shows that it is stored with the file as a block of bytes after the actual content of the file.
Is this the way of metadata storage for most of the other file types?
As far as I know, audio file formats :
May support metadata standards (e.g. ID3v1, ID3v2, APEtag, iXML)
May also have their own native metadata format (e.g. MP4 boxes / Quicktime atoms, OGG/FLAC/OPUS/Speex/Theora VorbisComment, WMA native metadata, AIFF / AIFC native metadata...)
=> In these two cases, metadata is stored directly into the audio file itself.
HydrogenAudio maintains a field mapping table between the most common formats : http://wiki.hydrogenaud.io/index.php?title=Tag_Mapping
That being said, many audio players (e.g. iTunes, foobar2000) allow their users to edit any metadata field in any file, regardless of whether said fields are supported or not by the underlying tagging standards (e.g. adding an "Album Artist" field in an S3M file).
In order to do that, these audio players store metadata in their internal database, thus giving the illusion that the audio file has been "enriched" while its actual content remain unchanged.
Another classic use of audio player databases is to store the following fields :
Rating
Number of times played
Last time played
=> In that case, you'll find metadata in the audio player's internal database
Related
Is there an audio file format, where I can save all the individual chunks (recorded in javascript) while splitting them up at any point to save them to different files and have them still all playable?
Yes this is what WAV file does ... if you save the file to conform to WAV payload format you can play back the file you create as a WAV file even without file having its normal 44 byte header
I store raw audio data in arrays that can be sent to Web Audio API's AudioBuffer. The raw audio data arrays can be manipulated as you wish.
Specifics for obtaining the raw data are going to vary from language to language. I've not obtained raw data from within JavaScript. My experience comes from generating the data algorithmically or from reading .wav files with Java's AudioInputLine, and shipping the data to JavaScript via Thymefeaf.
In Apache NiFi I can have an input with compressed data that's unpacked using the UnpackContent processor and then connect the output to further record processing or otherwise.
Is it possible to operate directly on the compressed input? In a normal programming environment, one might easily wrap the record processor in a container that more or less transparently unpacks the data in a stream-processing fashion.
If this is not supported out of the box, would it be reasonable to implement a processor that extends for example ConvertRecord to accept a compressed input?
The motivation for this is to work efficiently with large CSV data files, converting it into a binary record format without having to spill the uncompressed CSV data to disk.
Compressed input for record processing is not supported currently, but is a great idea for improvement.
Instead of implementing it at a particular processor (e.g. ConvertRecord), I'd suggest following two approaches:
Create CompressedRecordReaderFactory implementing RecordReaderFactory
Like Java compressed stream such as GZIPInputStreawm, CompressedRecordReaderFactory will wrap another RecordReaderFactory, user specifies compression type (or the reader factory may be able to implement auto-detect capability by looking at FlowFile attributes ... etc)
Benefit of this approach is once we add this, we can support reading compressed input stream at any existing RecordReader and Processors using Record api, not only CSV, but also XML, JSON ... etc
Wrap InputStream at each RecordReaderFactory (e.g. CSVReader)
We could implement the same thing at each RecordReaderFactory and supporting compressed input gradually
This may provide a better UX because no additional ControllerService has to be configured
How do you think? For further discussion, I suggest creating a NiFi JIRA ticket. If you're willing to contribute, that would be even better.
I am converting .ima files, collected by an audiologger, into .wav format. It works fine, but when doing this I loose the information about the date/time at which the (original, .ima) files were created. Is there a way of having the .wav files somehow 'timestamped' so I could recover the date/time at which the audio was recorded?
Many thanks for any hint provided.
As commented, you can either:
Store the date/time information in the file name
For example, store files with file names in the format 2018-09-23-19-53-45.wav, or whatever time format you like.
Store the audio in Broadcast WAV format files (BWF)
Broadcast WAV is based on WAV format but allows for metadata in the file. The difference between a Broadcast WAV file and a normal WAV is the presence of the BEXT chunk, and as such the file is compatible with existing WAV players.
The BEXT chunk contains two appropriate fields called OriginationDate and OriginationTime. The layout for the chunk can be found here: BEXT Audio Metadata Information.
I'm building an application that requires that a user be able upload audio files, and then at a later time request those same files. Being that I'm very new to all this, creating a file directory seems really confusing to me, GridFS ( storing the audio files in the database) seems to be easier to understand at this point.
What I am confused about is, if I go the direction of GridFS, does every user need to have a GridFS collection. Or would I somehow set up one main GridFS collection, and all user's audio files will be in that collection. Then in my mongoose user model I would save the names of the audio file's that belong to a given user. And then when the user requests their audio files, I will get the list of files that belong to that user, search the one main GridFS collection for those files?
I know that I might be better off setting up a file system(performance reasons), I looked into nginx, but I found myself just getting more and more confused.
Treat gridFS as a collection with audio files (so we have fileId and content),
then you need to store fileId somewhere with owner record/document.
What can be an issue that storing pointer to user files in one document can go beyond max document size (16MB) - so if this could be a case - then we need a simple userId-fileId collection.
Have a fun!
I have set of data which contains images also. I want to cache this data. Should i store them on file system or on core data and why?
There are two main options:
Store the file on disk, and then store the path to the image in core data
Store the binary data of the image in core data
I personally prefer the 1st option, since it allows me to choose when I want to load the actual image in memory. It also means that I don't have to remember what format the raw data is in; I can just use the path to alloc/init a new UIImage object.
You might want to read this from the Core Data Programming Guide on how to deal with binary large objects (BLOBs). There are rules of thumb for what size binary data should and should not be stored within the actual Core Data store.
You might also look at Core Data iPad/iPhone BLOBS vs File system for 20k PDFs
If you do place binary data within Core Data store, you would do well to have a "Data" entity that holds the actual data and to have your "Image" entity separate. Create a relationship between the two entities, so that "Data" need only be loaded when actually needed. The "Image" entity can hold the meta-data such as title, data type, etc.
With regards to where to store the user data/files (I found "application support" to be a decent location given that i was wary of the user moving, deleting or altering the file in some way that would result in the image not being able to be recovered and used later by my application)
Take minecraft as an example:
eg. "~/Library/Application Support/minecraft/saves/"
I would agree with the previous comments and store paths to the images in core data but otherwise store the images themselves as png files in their own folder outside of core data.