IMFMediaPlayer hangs during SetSourceFromByteStream - audio

Background: I'm coding a metro-styled app for Win8. I need to be able to play music-file. Because of quality and space requirements we're using encoded audio (mp3/ogg).
I'm using XAudio2 to play sound effects (.wav files), but since I couldn't figure out a way to play encoded audio with it, I decided to play the music files with Media Foundation (IMFMediaPlayer interface).
I downloaded metro apps sample, and found out that the Media Engine Native C++ video playback sample was closest to what I needed.
Now that my app has MediaPlayer playing musics, I ran into a problem. If the device running the app is slow enough, MediaPlayer hangs. When I'm running the release-version of the app on my device, it's fine and I can hear the music just fine. But when I attach the debugger or run it on a slower device, it hangs when I'm setting bytestream for the MediaPlayer to play.
Here's some code, you'll find it pretty similiar to the sample:
StorageFolder^ installedLocation = Windows::ApplicationModel::Package::Current->InstalledLocation;
m_pickFileTask = Concurrency::task<StorageFile^>(installedLocation->GetFileAsync(filename)), m_tcs.get_token());
auto player = this;
m_pickFileTask.then([player](StorageFile^ fileHandle)
{
player->SetURL(fileHandle->Path);
Concurrency::task<IRandomAccessStream^> fOpenStreamTask = Concurrency::task<IRandomAccessStream^> (fileHandle->OpenAsync(Windows::Storage::FileAccessMode::Read));
fOpenStreamTask.then([player](IRandomAccessStream^ streamHandle)
{
MEDIA::ThrowIfFailed(
player->m_spMediaEngine->Pause()
);
MEDIA::GetMediaError(player->m_spMediaEngine);
player->SetBytestream(streamHandle);
if (player->m_spMediaEngine)
{
MEDIA::ThrowIfFailed(
player->m_spEngineEx->Play()
);
MEDIA::GetMediaError(player->m_spMediaEngine);
}
}
);
}
);
And here's the SetBytestream method:
SetBytestream(IRandomAccessStream^ streamHandle)
{
if(m_spMFByteStream != nullptr)
{
m_spMFByteStream->Close();
m_spMFByteStream = nullptr;
}
MEDIA::ThrowIfFailed(
MFCreateMFByteStreamOnStreamEx((IUnknown*)streamHandle, &m_spMFByteStream)
);
MEDIA::ThrowIfFailed(
m_spEngineEx->SetSourceFromByteStream(m_spMFByteStream.Get(), m_bstrURL)
);
MEDIA::GetMediaError(m_spEngineEx);
return;
}
The line where it hangs is:
m_spEngineEx->SetSourceFromByteStream(m_spMFByteStream.Get(), m_bstrURL)
When I'm debugging the app, I can press pause and see the stack. Well, not much of it, but atleast I can see it that it's indefinitely at
ntdll.dll!77b7f4dc()
Any ideas why my app would hang in such a way?
(OPTIONAL: If you know a better way to play mp3/ogg in a c++ metro-styled app, let me know)

Could not figure out why this is happening, but I managed to code a work-a-round:
IMFSourceReader can be used to decode MP3s and feed bytes into XAudio2SourceVoice.
XAudio2 audio stream effect sample contains good example how to do this.

Related

Chrome "stalling" when streaming mp3 file from nodejs windows only

We've got a really annoying bug when trying to send mp3 data. We've got the following set up.
Web cam producing aac -> ffmpeg convert to adts -> send to nodejs server -> ffmpeg on server converts adts to mp3 -> mp3 then streamed to browser.
This works *perfectly" on Linux ( chrome with HTML5 and flash, firefox flash only )
However on windows the sound just "stalls", no matter what combination we use ( browser/html5/flash ). If however we shutdown the server the sound then immediately starts to play as we expect.
For some reason on windows based machines it's as if the sound is being buffered "waiting" for something but we don't know what that is.
Any help would be greatly appreciated.
Relevant code in node
res.setHeader('Connection', 'Transfer-Encoding');
res.setHeader('Content-Type', 'audio/mpeg');
res.setHeader('Transfer-Encoding', 'chunked');
res.writeHeader('206');
that.eventEmitter.on('soundData', function (data) {
debug("Got sound data" + data.cameraId + " " + req.params.camera_id);
if (req.params.camera_id == data.cameraId) {
debug("Sending data direct to browser");
res.write(data.sound);
}
});
Code on browser
soundManager.setup({
url: 'http://dashboard.agricamera.co.uk/themes/agricamv2/swf/soundmanager2.swf',
useHTML5Audio: false,
onready: function () {
that.log("Sound manager is now ready")
var mySound = soundManager.createSound({
url: src,
autoLoad: true,
autoPlay: true,
stream: true,
});
}
});
If however we shutdown the server the sound then immediately starts to play as we expect.
For some reason on windows based machines it's as if the sound is being buffered "waiting" for something but we don't know what that is.
That's exactly what's happening.
First off, chrome can play ADTS streams so if possible, just use that directly and save yourself some audio quality by not having to use a second lossy codec in the chain.
Next, don't use soundManager, or at least let it use HTML5 audio. You don't need the Flash fallback these days in most cases, and Chrome is perfectly capable of playing your streams. I suspect this is where your problem lies.
Next, try disabling chunked transfer. Many clients don't like transfer encoding on streams.
Finally, I have seen cases where Chrome's built-in media handling (which I believe varies from OS to OS) cannot sync to the stream. There are a few bug tickets out there for Chromium. If your playback timer isn't incrementing, this is likely your problem and you can simply try to reload the stream programmatically to work around it.

Windows Phone 8.1 play audio data stream through speaker?

I receive over network PCM audio data stream and this part works fine so I am ending up with
DataReader incomming = args.GetDataReader();
byte[] RcvBuffer = new byte[incomming.UnconsumedBufferLength];
incomming.ReadBytes(RcvBuffer);
I have all audio data in buffer.
How I can play this through telephone Speaker ? Can you point me in some direction ?
Thanks
There're many ways to do that.
You can prepend the WAVE header to your data, and use MediaElement for playback, see the documentation for SetSource method.
If however by “telephone speaker” you mean the earphone, then it is only possible if you are creating a VoIP app.
It took a while but I sorted it, maybe someone else will need help in the future.
First Problem - since I just started app development for Windows Phone I have chosen Blank App (Windows Phone) instead Blank App (Windows Phone Silverlight) and I did not have access to many features that are available in Silverlight projects, so my suggestions for beginners: understand what each project is for.
Like Soonts said there are many ways to do this, this is one that I used.
I simplified this code and retyped this so there can be some typos.
using Microsoft.Xna.Framework.Audio;
using System.IO;
1) Create Stream to load your incoming data:
MemoryStream stream = new MemoryStream();
2) Load data from buffer to stream:
stream.Write(RcvBuffer, 0, RcvBuffer.Length);
3) I am using SoundEfect to play this through Loud-Speaker. Sample rate that I use is 8 kHz
SoundEffect sound;
sound = new SoundEffect(stream.toArray(), 8000, AudioChannels.Mono)
sound.Play();

Best approach for playing a single tone audio file?

Can anyone tell me the best approach to playing single-tone, audio (.mp3) files in a Windows Phone 8 app? Think of a piano app, where each key would represent a button, and each button would play a different tone.
I'm looking for the most efficient way to go about this - I've got 8 different buttons that need to play a different tone when tapped.
I tried using the MediaElement:
MediaElement me;
public MainPage()
{
InitializeComponent();
me = new MediaElement();
me.AutoPlay = false;
me.Source = new Uri("/Sounds/Sound1.mp3", UriKind.Relative);
btnPlay.Click += btnPlay_Click;
}
private void btnPlay_Click(object sender, EventArgs e)
{
me.Play();
}
But nothing happens, either in the emulator or on a device (testing w/ a Lumia 822). Am I doing something wrong here? It seems like it should be pretty simple. Or would using MediaElement even be the best thing to use for my scenario?
Would this fall under the Background Audio category? I've read through this example but it seems overkill for what I want to do.
I've also read about using XNA's SoundEffect to do the job, but then I'd have to convert my .mp3 files to .wav (which isn't necessarily a problem, but I'd rather not go through that if I don't need to).
Can anyone tell me either what I'm doing wrong in my example above or guide me to a better solution for playing quick <1s audio tones?
I had this problem before with MediaElement not playing audio files. After many attempts I found out that it only plays if it defined in the xaml and AutoPlay is set to true.
Try defining it in the xaml or you can just add it to your LayoutRoot.
var me = new MediaElement();
LayoutRoot.Children.Add(me);
me.AutoPlay = true;
me.Source = new Uri("Sound/1.mp3", UriKind.Relative);
I have had good luck just doing this piece of code in my app. But it may not work as well in your context, give it a whirl though.
mediaElement.Source = new Uri("/Audio/" + songID.ToString() + ".mp3", UriKind.Relative);
mediaElement.Play();

Adobe AIR for iPhone app: can't play audio properly

I just developed an App by using adobe air. It contains some animations with background music in mp3 format. The problem is that the music is very jerky when the animation is playing...
FYI, this is the way how I play audio in flash:new Sound(new URLRequest("m3.mp3")).play()
Have I done anything wrong?
BTW, the funny thing is that if you hit the HOME button, and then come back to the app again, everything plays beautifully...
Without knowing more about the code, it seems like the sound is not fully loaded. The file plays as far as it can, then waits for more data to show up, then continues . . . very jerky. You may have to wait for the sound to load completely before playing it:
var s = new air.Sound();
s.addEventListener(air.Event.COMPLETE, onSoundLoaded);
var req = new air.URLRequest("bigSound.mp3");
s.load(req);
function onSoundLoaded(event)
{
var localSound = event.target;
localSound.play();
}
This code is from Adobe's Sound docs.

Blackberry Audio Recording Sample Code

Does anyone know of a good repository to get sample code for the BlackBerry? Specifically, samples that will help me learn the mechanics of recording audio, possibly even sampling it and doing some on the fly signal processing on it?
I'd like to read incoming audio, sample by sample if need be, then process it to produce a desired result, in this case a visualizer.
RIM API contains JSR 135 Java Mobile Media API for handling audio & video content.
You correct about mess on BB Knowledge Base. The only way is browse it, hoping they'll not going to change site map again.
It's Developers->Resources->Knowledge Base->Java API's&Samples->Audio&Video
Audio Recording
Basically it's simple to record audio:
create Player with correct audio encoding
get RecordControl
start recording
stop recording
Links:
RIM 4.6.0 API ref: Package javax.microedition.media
How To - Record Audio on a BlackBerry smartphone
How To - Play audio in an application
How To - Support streaming audio to the media application
How To - Specify Audio Path Routing
How To - Obtain the media playback time from a media application
What Is - Supported audio formats
What Is - Media application error codes
Audio Record Sample
Thread with Player, RecordControl and resources is declared:
final class VoiceNotesRecorderThread extends Thread{
private Player _player;
private RecordControl _rcontrol;
private ByteArrayOutputStream _output;
private byte _data[];
VoiceNotesRecorderThread() {}
private int getSize(){
return (_output != null ? _output.size() : 0);
}
private byte[] getVoiceNote(){
return _data;
}
}
On Thread.run() audio recording is started:
public void run() {
try {
// Create a Player that captures live audio.
_player = Manager.createPlayer("capture://audio");
_player.realize();
// Get the RecordControl, set the record stream,
_rcontrol = (RecordControl)_player.getControl("RecordControl");
//Create a ByteArrayOutputStream to capture the audio stream.
_output = new ByteArrayOutputStream();
_rcontrol.setRecordStream(_output);
_rcontrol.startRecord();
_player.start();
} catch (final Exception e) {
UiApplication.getUiApplication().invokeAndWait(new Runnable() {
public void run() {
Dialog.inform(e.toString());
}
});
}
}
And on thread.stop() recording is stopped:
public void stop() {
try {
//Stop recording, capture data from the OutputStream,
//close the OutputStream and player.
_rcontrol.commit();
_data = _output.toByteArray();
_output.close();
_player.close();
} catch (Exception e) {
synchronized (UiApplication.getEventLock()) {
Dialog.inform(e.toString());
}
}
}
Processing and sampling audio stream
In the end of recording you will have output stream filled with data in specific audio format. So to process or sample it you will have to decode this audio stream.
Talking about on the fly processing, that will be more complex. You will have to read output stream during recording without record commiting. So there will be several problems to solve:
synch access to output stream for Recorder and Sampler - threading issue
read the correct amount of audio data - go deep into audio format decode to find out markup rules
Also may be useful:
java.net: Experiments in Streaming Content in Java ME by Vikram Goyal
While not audio specific, this question does have some good "getting started" references.
Writing Blackberry Applications
I spent ages trying to figure this out too. Once you've installed the BlackBerry Component Packs (available from their website), you can find the sample code inside the component pack.
In my case, once I had installed the Component Packs into Eclipse, I found the extracted sample code in this location:
C:\Program
Files\Eclipse\eclipse3.4\plugins\net.rim.eide.componentpack4.5.0_4.5.0.16\components\samples
Unfortunately when I imported all that sample code I had a bunch of compile errors. To workaround that I just deleted the 20% of packages with compile errors.
My next problem was that launching the Simulator always launched the first sample code package (in my case activetextfieldsdemo), I couldn't get it to run just the package I am interested in. Workaround for that was to delete all the packages listed alphabetically before the one I wanted.
Other gotchas:
-Right click on the project in Eclipse and select Activate for BlackBerry
-Choose BlackBerry -> Build Configurations... -> Edit... and select your new project so it builds.
-Make sure you put your BlackBerry source code under a "src" folder in the Eclipse project, otherwise you might hit build issues.

Resources