Realtime 3D Audio Streaming and Playback - audio

Question

Currently, I am working on streaming audio to Unity over network, I successfully integrated media library (GStreamer) with Unity and I was able to play the audio inside the environment using audio filter callback function attached to AudioSource:
void OnAudioFilterRead(float[] data, int channels)
{
// fill data array with the streamed audio data
//....
}
The previous function provided 2D audio playback with very low latency,
In my application I want to render the audio in 3D Spatial space, so the audio rendering will be dependent on camera's (Audio listener) orientation.
I tried to stream audio data into AudioClip using the following:
AudioClip TargetClip;
public AudioSource TargetSrc;
void Start()
{
int freq=32000; //streamed audio sampling rate
TargetClip = AudioClip.Create ("test_Clip", freq, 1, freq, true,true, OnAudioRead,OnAudioSetPosition);
TargetSrc.clip = TargetClip;
TargetSrc.Play ();
}
void OnAudioRead(float[] data) {
// fill data array with the streamed audio data
//....
}
void OnAudioSetPosition(int newPosition) {
}
When I played the audio, the audio was rendered as I wanted in 3D spatial space, however there was a huge latency (more than 2 seconds).
Is there any way to solve the latency problem?

Answer 1

I figured out how to solve this issue.
For those facing similar problem, actually the method I was using in the first place was not efficient by filling the AudioClip data. I was supposed to use OnAudioFilterRead() in either cases, however it should be using as following for spatial audio calculations:
public AudioSource TargetSrc;
void Start()
{
var dummy = AudioClip.Create ("dummy", 1, 1, AudioSettings.outputSampleRate, false);
dummy.SetData(new float[] { 1 }, 0);
TargetSrc.clip = dummy; //just to let unity play the audiosource
TargetSrc.loop = true;
TargetSrc.spatialBlend=1;
TargetSrc.Play ();
}
void OnAudioFilterRead(float[] data, int channels)
{
// "data" contains the weights of spatial calculations ready by unity
// multiply "data" with streamed audio data
for(int i=0;i<data.Length;++i)
{
data[i]*=internal_getSample(i);
}
}
Hope this would help some else.