How to play overlapping audio in winrt? - audio

I'm porting an app from wp8 that requires playback of various sounds that can overlap. The only way I've found so far it to use MediaElement, but this doesn't allow overlapping sounds.
QUESTION - what is the easiest and best audio engine to use to play overlapping audio? Ideally I need a small example of how I can do this.
I've looked into WASAPI (http://code.msdn.microsoft.com/windowsapps/Windows-Audio-Session-22dcab6b), but it doesn't look like it supports simple playback ?
Maybe I can wrap the MediaFoundation and call it from winrt? (MediaEngine audio playback on WinRT)
Here is my code now, but when I play a new sound it cuts off the previously playing one rather than blending them.
ThreadUtility.runOnUiThread(
async delegate()
{
// TODO doesn't allow sounds to overlap!
Uri uri = new Uri(R.base_uri, R.raw.URI_PREFIX + resourceId);
StorageFile storageFile =
await Windows.Storage.StorageFile.GetFileFromApplicationUriAsync(
uri);
MediaElement element = new MediaElement();
var randomAccessStream = await storageFile.OpenReadAsync();
element.SetSource(randomAccessStream, storageFile.ContentType);
element.Volume = volume;
element.PlaybackRate = pitch;
//TODO element.Pan = pan;
element.Play();
}
);
SOLUTION (as per Filip's answer):
in the page class:
var mediaElements = new LinkedList<MediaElement>();
{
for (int channel = 0; channel < TeacherSoundGroover.NUM_CHANNELS; channel++)
{
var mediaElement = new MediaElement();
mediaElements.add(mediaElement);
// Must be in the tree otherwise it won't overlap!
m_titlePanel.Children.Add(mediaElement);
}
}
m_soundPlayer = new MySoundPlayer(mediaElements);
}
in the MySoundPlayer class:
ThreadUtility.runOnUiThread(
async delegate()
{
Uri uri = new Uri(R.base_uri, R.raw.URI_PREFIX + resourceId);
StorageFile storageFile =
await Windows.Storage.StorageFile.GetFileFromApplicationUriAsync(
uri);
if(m_mediaElements != null)
{
int count = m_mediaElements.size();
if (count > 0)
{
int channel = m_nextMediaElementToUse % count;
m_nextMediaElementToUse++;
MediaElement element = m_mediaElements.get(channel);
var randomAccessStream = await storageFile.OpenReadAsync();
element.Stop();
element.DefaultPlaybackRate = rate;
element.SetSource(randomAccessStream, storageFile.ContentType);
element.Volume = volume;
element.Balance = pan;
element.Play();
}
}
}
);

The easiest thing to do is use multiple MediaElement controls, though that might not give you desired results. The best way is to use XAudio2 either directly or through SharpDX if you want to avoid creating a C++/CX WinRT component.

Related

Flutter - how do I get the duration of a picked video file on Web?

I've tried everything and failed to determine the duration of a picked video file on Flutter Web. All libraries on pub.dev need a 'File' and this is not available on the web.
I've not been able to get this from the metadata either.
What worked for me, though I am unhappy with the solution approach is:
Widget buildHiddenVideoPlayer(Key key) {
var _videoElement = html.VideoElement();
_videoElement.id = 'htmlHiddenVideoID';
_videoElement.autoplay = true;
_videoElement.muted = true;
_videoElement.loop = true;
_videoElement.controls = false;
_videoElement.defaultMuted = true;
if (fileData != null) {
_videoElement.src = xfile.path; // where xfile is the file picked as XFile
}
// ignore: undefined_prefixed_name
ui.platformViewRegistry.registerViewFactory(
'htmlHiddenVideoID',
(int viewId) => _videoElement,
);
return HtmlElementView(
key: key,
viewType: 'htmlHiddenVideoID',
);
}
This widget is hidden in a 5 x 5 sized box behind a iFrame widget (in my implementation)
Then when I need the duration of a picked file:
VideoElement element = document.getElementById('htmlHiddenVideoID') as VideoElement;
setState(() {
element.src = pickedFile.path;
});
while (true) {
await Future.delayed(const Duration(milliseconds: 200), () {});
duration = element.duration;
if (!duration.isNaN) break; // duration is not returned immediately in many cases
}
element.pause(); // stops the hidden video from playing and consuming resources

Flutter (Dart): Get/Record audio stream from microphone and play it back immediately (real-time)

I need to be able to capture a stream of audio from the microphone and then pass it as argument or read it immediately in order to play it back as audio. To implement this in any other framework there are excellent tools and functions you can use but I need to archive that functionality on Flutter.
Any help or suggestions?
Please try this package flutter_sound.
https://github.com/dooboolab/flutter_sound
Here is reference link
https://medium.com/flutterpub/flutter-sound-plugin-audio-recorder-player-e5a455a8beaf
Creating instance.
FlutterSound flutterSound = new FlutterSound();
Starting recorder with listener.
String path = await flutterSound.startRecorder(null);
print('startRecorder: $path');
_recorderSubscription = flutterSound.onRecorderStateChanged.listen((e) {
DateTime date = new DateTime.fromMillisecondsSinceEpoch(e.currentPosition.toInt());
String txt = DateFormat('mm:ss:SS', 'en_US').format(date);
});
Stop recorder
String result = await flutterSound.stopRecorder();
print('stopRecorder: $result');
if (_recorderSubscription != null) {
_recorderSubscription.cancel();
_recorderSubscription = null;
}
Start player
String path = await flutterSound.startPlayer(null);
print('startPlayer: $path');
_playerSubscription = flutterSound.onPlayerStateChanged.listen((e) {
if (e != null) {
DateTime date = new DateTime.fromMillisecondsSinceEpoch(e.currentPosition.toInt());
String txt = DateFormat('mm:ss:SS', 'en_US').format(date);
this.setState(() {
this._isPlaying = true;
this._playerTxt = txt.substring(0, 8);
});
}
});

How to switch to Front Camera in Windows Phone 8.1 (WinRT/Jupiter)

I can't seem to find the property for the MediaCapture class that allows me to detect the front camera and switch to it if available. Here is my current setup of the device, it all works as expected on Windows (front cam) and Phone (rear cam). None of the Microsoft samples show the front camera being used in Universal or WP 8.1 (WinRT/Jupiter).
mediaCaptureManager = new MediaCapture();
await mediaCaptureManager.InitializeAsync();
if (mediaCaptureManager.MediaCaptureSettings.VideoDeviceId != "" && mediaCaptureManager.MediaCaptureSettings.AudioDeviceId != "")
{
StartStopRecordingButton.IsEnabled = true;
TakePhotoButton.IsEnabled = true;
ShowStatusMessage("device initialized successfully!");
mediaCaptureManager.VideoDeviceController.PrimaryUse = CaptureUse.Video;
mediaCaptureManager.SetPreviewRotation(VideoRotation.Clockwise90Degrees);
mediaCaptureManager.SetRecordRotation(VideoRotation.Clockwise90Degrees);
mediaCaptureManager.RecordLimitationExceeded += RecordLimitationExceeded;
mediaCaptureManager.Failed += Failed;
}
There is a sample on the Microsoft github page that is relevant, although they target Windows 10. Still, the APIs should work on 8/8.1.
UniversalCameraSample: This one does capture photos, and supports portrait and landscape orientations. Here is the relevant part:
private static async Task<DeviceInformation> FindCameraDeviceByPanelAsync(Windows.Devices.Enumeration.Panel desiredPanel)
{
// Get available devices for capturing pictures
var allVideoDevices = await DeviceInformation.FindAllAsync(DeviceClass.VideoCapture);
// Get the desired camera by panel
DeviceInformation desiredDevice = allVideoDevices.FirstOrDefault(x => x.EnclosureLocation != null && x.EnclosureLocation.Panel == desiredPanel);
// If there is no device mounted on the desired panel, return the first device found
return desiredDevice ?? allVideoDevices.FirstOrDefault();
}
And you can use it like so:
// Attempt to get the front camera if one is available, but use any camera device if not
var cameraDevice = await FindCameraDeviceByPanelAsync(Windows.Devices.Enumeration.Panel.Front);
if (cameraDevice == null)
{
Debug.WriteLine("No camera device found!");
return;
}
// Create MediaCapture and its settings
_mediaCapture = new MediaCapture();
var settings = new MediaCaptureInitializationSettings { VideoDeviceId = cameraDevice.Id };
// Initialize MediaCapture
try
{
await _mediaCapture.InitializeAsync(settings);
_isInitialized = true;
}
catch (UnauthorizedAccessException)
{
Debug.WriteLine("The app was denied access to the camera");
}
catch (Exception ex)
{
Debug.WriteLine("Exception when initializing MediaCapture with {0}: {1}", cameraDevice.Id, ex.ToString());
}
Have a closer look at the sample to see how to get all the details. Or, to have a walkthrough, you can watch the camera session from the recent //build/ conference, which includes a little bit of a walkthrough through some camera samples.
Here is how to get the device's available cameras and set the front one for the stream:
mediaCaptureManager = new MediaCapture();
var devices = await DeviceInformation.FindAllAsync(DeviceClass.VideoCapture);
var deviceInfo = devices[0]; //grab first result
foreach (var device in devices)
{
if (device.Name.ToLowerInvariant().Contains("front"))
{
deviceInfo = frontCamera = device;
hasFrontCamera = true;
}
if (device.Name.ToLowerInvariant().Contains("back"))
{
rearCamera = device;
}
}
var mediaSettings = new MediaCaptureInitializationSettings
{
MediaCategory = MediaCategory.Communications,
StreamingCaptureMode = StreamingCaptureMode.AudioAndVideo,
VideoDeviceId = deviceInfo.Id
};
await mediaCaptureManager.InitializeAsync(mediaSettings);
You'll need to consider rotation because front and rear cameras on different devices have different rotations, but this will initialize your MediaCapture properly

Add a Sound Mute Button to a Video Object

In Flash ActionScript 3.0. I have added a video Object to the stage by attaching the camera and video classes. I have also created a simple mute/unmute button. I want to embed this sound button into the video object to give clients more options. Is there anyway I can incorporate the mute button into the video object? This is my code: I wont display it all its too but any help here be much appreciated!!
var nc:NetConnection = new NetConnection();
nc.addEventListener(NetStatusEvent.NET_STATUS, netHandler);
nc.connect(rtmfp://example);
cam:Camera = Camera.getCamera();
var vid:Video = new Video();
camera.setMode(720,540,15,true);
camera.setQuality(0, 100);
vid.attachCamera(cam);
video.width = camera.width;
video.height = camera.height;
addChild(vid);
function setMute(vol){
var sTransform:SoundTransform = new SoundTransform(1,0);
sTransform.volume = vol;
SoundMixer.soundTransform = sTransform;
}
var isMuted:Boolean = false;
mute_Btn.addEventListener(MouseEvent.CLICK, toggleMuteBtn);
function toggleMuteBtn(event:MouseEvent):void{
if(isMuted){
isMuted = false;
setMute(1);
} else {
isMuted = true;
setMute(0);
Once you have added the video to the stage, you can do this;
video.soundTransform = new SoundTransform(0.5);
or in your situation, something like...
function setMute(vol)
{
root.video.soundTransform = new SoundTransform(vol);
}

How to get an existing video thumbnail image from MonoTouch?

I'm trying to get a thumbnail image for a video that I've just chosen (or recorded) via the UIImagePickerController. In my picker's FinishedPickingMedia method, I get what I believe to be the path to the video file and use it to save the recording to album. The newly recorded video does indeed appear in my album. Now to get the thumbnail…
I've tried using the ALAssetsLibrary in a couple ways. First, I've enumerated all the videos and tried to find a match from the asset's UtiToUrlDictionary.Values[0] value, which reveals its file path. No entries matched the path I obtained from the FinishedPickingMedia method's mediaUrlKey.
My second attempt was to use the ALAssetsLibrary's AssetForUrl() method. Again, I tried my FinishedPickingMedia method's mediaUrlKey, but there was no match.
The UIImagePickerController -> FinishedPickingMedia -> mediaUrlKey returns a path of:
file://localhost/private/var/mobile/Applications/73037738-C060-4DE6-A1CA-698E6BE083F2/tmp//trim.iRBwWw.MOV
After painful digging and inspecting every video in my library, it the same video's path from the ALAssetsLibrary is:
assets-library://asset/asset.MOV?id=B073FCF6-83ED-436B-AFD8-42D0A6C70FC6&ext=MOV
Why are the video paths different between UIIMagePickerController and ALAssetsLibrary? How can I get ALAssetsLibrary to return the same video picked from UIImagePickerController?
[code for the picker]
public override void FinishedPickingMedia (UIImagePickerController picker, NSDictionary info)
{
var mediaUrlKey = new NSString("UIImagePickerControllerMediaURL");
var mediaPath = (NSUrl) info.ObjectForKey(mediaUrlKey);
if( recordView.RecordVideo )
{
if(mediaPath != null)
{
if(UIVideo.IsCompatibleWithSavedPhotosAlbum(mediaPath.Path))
{
UIVideo.SaveToPhotosAlbum(mediaPath.Path, SaveToPhotosAlbumResults);
}
else
{
using (var alert = new UIAlertView("Problem Encountered",
"Unable to save this recording.", null, "Ok!", null))
{
alert.Show();
}
}
}
}
else
{
recordView.VideoFileLocation = mediaPath;
//ALAssetsLibrary library = new ALAssetsLibrary();
//library.AssetForUrl( mediaPath, GetAssetResult, GetAssetError );
}
recordView.VideoFileLocation = mediaPath;
Console.WriteLine("***" + recordView.VideoFileLocation.ToString()+ "***");
recordView.ShowUploadButton();
recordView.SetThumbnailImage();
picker.DismissModalViewControllerAnimated(true);
}
Thank you
I think may be it is too late but the following code work as of now
public static UIImage GenerateImage(NSUrl videoUrl)
{
var asset = AVAsset.FromUrl(videoUrl);
var imageGenerator = AVAssetImageGenerator.FromAsset(asset);
imageGenerator.AppliesPreferredTrackTransform = true;
var actualTime = asset.Duration;
CoreMedia.CMTime cmTime = new CoreMedia.CMTime(1, 60);
NSError error;
var imageRef = imageGenerator.CopyCGImageAtTime(cmTime, out actualTime, out error);
if (imageRef == null)
return null;
var image = UIImage.FromImage(imageRef);
return image;
}
I think the section Can I Get To The Original File Representing This Image on the Disk in this question can help you.
This is how you can get a thumbnail within the picker delegate:
NSUrl data = new NSUrl(info.ObjectForKey(new NSString("UIImagePickerControllerMediaURL")).ToString());
//get the video thumbnail
MPMoviePlayerController movie = new MPMoviePlayerController(data);
movie.Stop();
UIImage videoThumbnail = movie.ThumbnailImageAt(0.0,MPMovieTimeOption.NearestKeyFrame);
movie.Stop();

Resources