now i'm working on a project for creating audio unit instrument that provide the basic waveform and also provide the audio sampler. i have a problem with how to implement the audio unit instrument base to support the audio file browsing and also wonder about the Audio unit SDK that support this situation to making a sampler.
the sampler can combine with wave form then generate the new sound
This is not an IOS audio unit. and i have not much knowledge about this sampler structure
i have been search a lot, but their are no related knowledge and some source code that i can understand. pls help me for at least browsing the audio file from Au Instrument and slicing the audio data in a time domain. so i can use DSP to work with it.
regard.
I suggest taking a look at the FilterDemo source code. It illustrates the most important aspects of the relationship between parameters, properties, UI, and the underlying DSP code. I have had some success with using the FilterDemo source code as a basis for converting raw DSP code, as well as AU plugins with only generic parameters (and therefore no UI), into fully integrated AU plugins with customized UI.
Also, pay close attention to the warnings, embedded in some of the source code, about renaming your UI elements, as there is a flat namespace to contend with.
Related
is there a scenario where we can use the Google Resonance Audio SDK not with headphones, but with real speakers (e.g. mounted in a 360° cyrcle setting)?
Or are all algorithms not working for real speaker outputs?
Thank you!
Currently, Resonance Audio is optimized for headphone playback. For example, HRTF processing is done in the Ambisonics domain, without generating (virtual) speaker signals - this is because it is a much more efficient way of generating binaural output.
However, in the Resonance Audio open source release, the Ambisonic Codec class can readily be used to decode Ambisonics to any arbitrary loudspeaker array. To use that with the rest of the Resonance Audio system, however, it would be necessary to modify/extend the audio processing graph by adding a new decoder node.
Please, feel free to add a feature request and, depending on popularity, we might consider adding that in the future!
I have been tasked to tag a video frame-by-frame with gps coordinates as it is recording.
The platform must be on Linux (Ubuntu to be specific).
Very new to programming with video sources..
Some questions :
Do video frames even have per-frame meta data?
Is GStreamer a good framework to use for my purposes? How should I get started?
Thanks.
Check GstMeta: http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/gstreamer-GstMeta.html
It allows you to attach arbitrary metadata to buffers, which then can be passed downstream with the buffers and passed through other elements if possible. Take a look at the code of existing GstMeta implementations in gst-plugins-base for examples: http://cgit.freedesktop.org/gstreamer/gst-plugins-base/tree/gst-libs/gst/video/gstvideometa.h http://cgit.freedesktop.org/gstreamer/gst-plugins-base/tree/gst-libs/gst/video/gstvideometa.c
Your meta would probably work very similar to the region of interest meta (plain metadata)
To get started, read the documentation on http://gstreamer.freedesktop.org , especially start with the application writers manual. And take a look at existing GStreamer code to understand how everything works together.
I'm looking for an audio processing language or library which will allow me to experiment with different synthesis techniques. I've looked at Processing which I think is great at what it does, but haven't found any inspiring (and simple) audio libraries.
As a baseline, I want to simply create my own sample buffers and play them back (ideally in realtime). As a plus, the ability to handle MIDI events would be great. I'm an experienced C++ programmer so I could do it natively on but had hoped there was a more DSL (domain specific language) approach.
I have access to Windows, Mac or Linux so not too bothered yet about platform. Other languages I can deal with are C#, Java & Python.
Thanks
James
Depending on how much you want to stay out of the low-level housekeeping details, you may want to look at CSound , or if you want to not actually write code, the patching-based system PureData is great to work with. As #Lou points out, ChucK is interesting (but was too buggy to use the last time I checked it out).
If you really do want to write code, look at the Synthesis Toolkit, a set of C++ classes for audio processing and synthesis.
For an app framework, I recommend JUCE, which has incredibly nice cross-platform handling of audio/midi IO and GUI elements.
Max MSP is an audio production tool that is highly expressive.
I guess you could say it's a high-level tool, and not a low-level programming language. My impression of it is that it's geared towards the technical musician or the artistic engineer, but anyway it kicks ass and you could go low-level with it if you want.
I've always been a big fan of SuperCollider. It's designed for Mac OS X but also works on Linux.
The language is mostly based on SmallTalk, and it's pretty easy to pick up if you understand the basics of functional programming. The quality of the sound output by the SC Server is very good and there is plenty of documentation both built into the app environment and available online.
One interesting point of SuperCollider is the usage on android devices, and it's intercommunication with python trough out other modules.
Here goes an example
I know you didn't say Ruby, but check out Archaeopteryx
https://github.com/gilesbowkett/archaeopteryx/wiki
or ChucK
http://chuck.cs.princeton.edu/
Have a look at NAudio, an open source .NET audio SDK for working with audio files and devices in Windows. Some features include:
http://naudio.codeplex.com/
NAudio Features:
Play back audio using a variety of APIs
Decompress audio from different Wave Formats
Record audio using WaveIn, WASAPI or ASIO
Read and Write standard .WAV files
Mix and manipulate audio streams using a 32 bit floating mixing engine
Extensive support for reading and writing MIDI files
Full MIDI event model
Basic support for Windows Mixer APIs
A collection of useful Windows Forms Controls
Some basic audio effects, including a compressor
I've searched the net but didn't found anything interesting. Maybe I'm doing something wrong.
I'm looking for sound synthesis API written on C, C++ or even Objective-C, which can synthesize different types of waves, effects are optional.
Here's a complete library/toolkit for FM (Frequency Modulation) synthesis:
link1
link2
If you have time to spare... creating simple sound synthesis from scratch is actually a fun endeavor. If you create a small buffer of 256 16 bit samples which represent either a sine. a sawtooth, block or pulse, you can copy these to a live audiobuffer (e.g. a small buffer (say 16kb)) which constantly loops. By staying ahead of the playposition, and constantly filling up the buffer with new values, you can create the soundoutput.
You can use the small buffers to combine these in interesting ways (simplest is just to add them together (additive synthesis)).
The frequency of the tone can be manipulated by using a bigger or smaller sampling step through the small buffers. Amplitude can be manipulated by scaling the samples before putting them into the output buffer.
Great fun experimenting with this!
If you have this step nailed, you can add more sophisticated effects like filters (low pass, high pass, etc) and effects (reverbs, echoes, etc)
R
Have you looked at the synthesis toolkit (STK)? It's in C++ (I don't think ObjC is the right language for audio synthesis, in fact audio units, Apple's own way of doing audio stuff, including generators/filters/effects... is in C++).
STK will run on Mac OS X, and iOS no problem (CoreAudio is supported), but will also run on Linux and Windows (Direct sound and ASIO), using RtAudio. It's really nicely done and lightweight, these guys have spent a lot of time thinking about it and it will definitely give you a big head start. It can handle loads of different audio file formats + midi (and hopefully OSC soon...).
There is also Create and CLAM which is huge, these include GUI components and many other things which you might or might not want. If you're only interested in doing sound synthesis I really recommend STK.
PortAudio is also a great C API that we used last semester in an audio programming course. It provides an audio callback...what more could you need!?
I haven't tried incorporating it with anything in Objective-C yet, but will report back when I do.
Writing audio synthesis algorithms in C/obj-C is quite difficult in my opinion. I would recommend writing your signal processing algorithms using PureData and then use ZenGarden or libpd to embed and interpret the pd patches in your app.
Another C++ library is nsound:
http://nsound.sourceforge.net
One can generate any kind of modulated signal using the Generator class or using the provided Sine class. Each time-step can have it's own instantaneous frequency and phase offset.
You can also experiment with the Python module to prototype your algorithm quickly, then implement in C++. It can produce pretty matplotlib plots from Python and even from C++!
Have you looked at CSound? It's an incredibly flexible audio generation platform, and can handle everything from simple waveform generation to FM synthesis and all kinds of filters. It also provides MIDI support, and you can extend it by writing custom opcodes. There's a full C API and several C++ APIs as well.
I'm wondering what is the recommended audio library to use?
I'm attempting to make a small program that will aid in tuning instruments. (Piano, Guitar, etc.). I've read about ALSA & Marsyas audio libraries.
I'm thinking the idea is to sample data from microphone, do analysis on chunks of 5-10ms (from what I've read). Then perform a FFT to figure out which frequency contains the largest peak.
This guide should help. Don't use ALSA for your application. Use a higher level API. If you decide you'd like to use JACK, http://jackaudio.org/applications has three instrument tuners you can use as example code.
Marsyas would be a great choice for doing this, it's built for exactly this kind of task.
For tuning an instrument, what you need to do is to have an algorithm that estimates the fundamental
frequency (F0) of a sound. There are a number of algorithms to do this, one of the newest and best
is the YIN algorithm, which was developed by Alain de Cheveigne. I recently added the YIN algorithm
to Marsyas, and using it is dead simple.
Here's the basic code that you would use in Marsyas:
MarSystemManager mng;
// A series to contain everything
MarSystem* net = mng.create("Series", "series");
// Process the data from the SoundFileSource with AubioYin
net->addMarSystem(mng.create("SoundFileSource", "src"));
net->addMarSystem(mng.create("ShiftInput", "si"));
net->addMarSystem(mng.create("AubioYin", "yin"));
net->updctrl("SoundFileSource/src/mrs_string/filename",inAudioFileName);
while (net->getctrl("SoundFileSource/src/mrs_bool/notEmpty")->to<mrs_bool>()) {
net->tick();
realvec r = net->getctrl("mrs_realvec/processedData")->to<mrs_realvec>();
cout << r(0,0) << endl;
}
This code first creates a Series object that we will add components to. In a Series, each of the components
receives the output of the previous MarSystem in serial. We then add a SoundFileSource, which you can feed
in a .wav or .mp3 file into. We then add the ShiftInput object which outputs overlapping chunks of audio, which
are then fed into the AubioYin object, which estimates the fundamental frequency of that chunk of audio.
We then tell the SoundFileSource that we want to read the file inAudioFileName.
The while statement then loops until the SoundFileSource runs out of data. Inside the while
loop, we take the data that the network has processed and output the (0,0) element, which is the
fundamental frequency estimate.
This is even easier when you use the Python bindings for Marsyas.
http://clam-project.org/
CLAM is a full-fledged software framework for research and application development in the Audio and Music Domain. It offers a conceptual model as well as tools for the analysis, synthesis and processing of audio signals.
They have a great API, nice GUI and a few finished apps where you can see everything.
ALSA is sort of the default standard for linux now by virtue of the kernel drivers being included in the kernel and OSS being depreciated. However there are alternatives to ALSA userspace, like jack, which seems to be aimed at low-latency professional type applications. It's API seems to have a nicer API, although I've not used it, my brief exposure to the ALSA API would make me think that almost anything would be better.
Audacity includes a frequency plot feature and has built-in FFT filters.