I am using MediaCapture to capture audio in my case from a mic and save it as mp3. It works, but the sound quality seems to be very poor when compared to an audio file recorded at the same time (same input) by any other programs like the wavepad. So my question is, is there any noise reduction technique or some other way to improve the audio quality captured through MediaCapture? Please note, I am already using AudioEncodingQuality.High in the CreateMp3 method, as shown in the following code.
var localFolder = Windows.Storage.ApplicationData.Current.LocalFolder;
StorageFile file = await localFolder.CreateFileAsync("audio.mp3", CreationCollisionOption.GenerateUniqueName);
_mediaRecording = await mediaCapture.PrepareLowLagRecordToStorageFileAsync(
MediaEncodingProfile.CreateMp3(AudioEncodingQuality.High), file);
await _mediaRecording.StartAsync();
Here are the recorded files by both apps.
Related
I am trying to build a web-application with the functionality of screen-recording with system audio + headphone-mic audio being captured in the saved video.
I have been thoroughly googling on a solution for this, however my findings show multiple browser solutions where the above works so long as headphones are NOT connected, meaning the microphone input is coming from the system rather than headset.
In the case that you connect headphones, all of these solutions capture the screen without video-audio, and the microphone audio from my headset. So to re-clarify on this, it should have recorded video-audio from the video being played whilst recording, and the headset-mic audio also.
This is thoroughly available in native applications, however I am searching for a way to do this on a browser.
If there are no solutions for this currently that anybody knows of, some insight on the limitations around developing this would also really help, thank you.
Your browser manages the media input being received in the selected tab/window
To receive media input, you need to ensure you have the checkbox Share Audio in the image below checked. However this will only record media-audio being played in your headphones, when it comes to receiving microphone audio, the opposite must be done i.e the checkbox should be unchecked, or merge the microphone audio separately on saving the recorded video
https://slack-files.com/T1JA07M6W-F0297CM7F32-89e7407216
create two const, one retrieving on-screen video, other retrieving audio media:
const DISPLAY_STREAM = await navigator.mediaDevices.getDisplayMedia({video: {cursor: "motion"}, audio: {'echoCancellation': true}}); // retrieving screen-media
const VOICE_STREAM = await navigator.mediaDevices.getUserMedia({ audio: {'echoCancellation': true}, video: false }); // retrieving microphone-media
Use AudioContext to retrieve audio sources from getUserMedia() and getDisplayMedia() separately:
const AUDIO_CONTEXT = new AudioContext();
const MEDIA_AUDIO = AUDIO_CONTEXT.createMediaStreamSource(DISPLAY_STREAM); // passing source of on-screen audio
const MIC_AUDIO = AUDIO_CONTEXT.createMediaStreamSource(VOICE_STREAM); // passing source of microphone audio
Use the method below to create a new audio source which will be used as as the merger or merged version of audio, then passing audios into the merger:
const AUDIO_MERGER = AUDIO_CONTEXT.createMediaStreamDestination(); // audio merger
MEDIA_AUDIO.connect(AUDIO_MERGER); // passing media-audio to merger
MIC_AUDIO.connect(AUDIO_MERGER); // passing microphone-audio to merger
Finally, connect the merged-audio and video together into one array to form a track, and pass it to the MediaStreamer:
const TRACKS = [...DISPLAY_STREAM.getVideoTracks(), ...AUDIO_MERGER.stream.getTracks()] // connecting on-screen video with merged-audio
stream = new MediaStream(TRACKS);
I am following the samples for Microsoft Cognitive Services Speech SDK, namely the Speech Translation.
The sample for dotnet core uses microphone as audio input and translates what you speak. Translated results are also available as synthesized speech. I would like to play this audio but could not find the appropriate code for that.
Tried using NAudio as sugguested in this answer but I get garbled audio. Guess there is more to the format of the audio.
Any pointers?
On .Net Core, many audio pacakges might not work. For example with NAudio, I can't play sound on my Mac.
I got it working using NetCoreAudio package (Nuget), with the following implementation in the translation Synthesizing event:
recognizer.Synthesizing += (s, e) =>
{
var audio = e.Result.GetAudio();
Console.WriteLine(audio.Length != 0
? $"AudioSize: {audio.Length}"
: $"AudioSize: {audio.Length} (end of synthesis data)");
if (audio.Length > 0)
{
var fileName = Path.Combine(Directory.GetCurrentDirectory(), $"{DateTime.Now.ToString("yyyy-MM-dd_HH-mm-ss.wav")}");
File.WriteAllBytes(fileName, audio);
var player = new Player();
player.Play(fileName).Wait();
}
};
in UWP application, Sometime Playing sound is stop.
await Execute.OnUIThreadAsync(async () =>
{
var element = new MediaElement();
var uri = new Uri($"ms-appx:///Assets/sound/abc.wav");
StorageFile sf = await StorageFile.GetFileFromApplicationUriAsync(uri);
var stream = await sf.OpenAsync(Windows.Storage.FileAccessMode.Read);
element.SetSource(stream, "");
element.Play();
});
I think, This UIThread job finish immediately.
but My sound file has 1 minutes length, Then,
The task was closed. then, Sound can not play by end.
How should I write to play sound ?
Refer to the following MSDN doc:Play media in the background. To support your music playing in background, you need to check the requirements from "Requirements for background audio". Actually you've mentioned that it is "sometime", so I'm not so sure whether you've already used the solution from the above doc. But if you haven't, you need to refer to that article, enable capbility and then manage both the transitioning and also notice the memory.
I have C# project where stream from ip-camera recorded to the file, I use libvlc.
This is part of code with vlc parameters:
string VlcArguments = #":sout=#transcode{acodec=mpga,deinterlace}:standard{access=file,mux=mp4,dst="C:\Users\I\Desktop\Output.mp4"}";
var media = factory.CreateMedia<IMedia>(rtsp://184.72.239.149/vod/mp4:BigBuckBunny_175k.mov, VlcArguments);
var player = factory.CreatePlayer<IPlayer>();
player.Open(media);
filename is the path of the result file.
It works fine, but I need to record sound from a microphone Microphone (High Definition Audio Device).
What I need to change to achieve that?
UPD
It should look something like this
var media = factory.CreateMedia<IMedia>("dshow:// dshow-vdev=rtsp://184.72.239.149/vod/mp4:BigBuckBunny_175k.mov dshow-adev=Microphone (High Definition Audio Device)", VlcArguments)
But it doesn't work (
UPD2
So, I think I found the answer
https://forum.videolan.org/viewtopic.php?f=14&t=124229&p=425550&hilit=camera+microphone+dshow#p425550
Unfortunately this will not work
I have a set of animations which I can make in Canvas (fabric.js) or WebGL (three.js). I need to record them automatically, server-side, through a script and output a video file.
The animations include:
Pictures
Videos (with audio)
Other animations/effects
I have researched a lot during last few months on this.
Results
1. Use PhantomJS + FFMPEG
Run HTML Canvas animations on headless browser(PhantomJS) and record with FFMPEG. Here the issue is PhantomJS supports neither WebGL nor Video element. http://phantomjs.org/supported-web-standards.html
2. Use Websockets to send data back to server using DataURL
Here again, we will need to run the animations on browser (which we can't because we have to do everything on server).
3. Use node-canvas
This is a library by TJ Holowaychuk which allows rendering HTML Canvas on Node.js. But it has its own limitations plus I haven't really explored this field much.
(If someone could shed more light on this library)
If anyone has done it before or can guide me somewhere useful.
All we need to do is use some data to create animations and record it into a video, everything on server side.
You can use electron to render WebGL pages with BrowserWindow option "show" set to false and/or use xvfb-run to run headless.
I don't think node-canvas supports the webgl context, so you'll
have to use a library built around 2d drawing, and it certainly
won't have support for any video codecs.
If you can get your animation to work using node-canvas, you can
grab the animated frames at a rate appropriate for your content,
something like this:
Disclosure: I've successfully used FFmpeg to encode a sequence
of externally generated images, but haven't tried the setInterval()
method below. In addition to the animation overhead itself, I don't
know how exporting a canvas to PNG files at 30 FPS would perform.
// assuming "canvas" is asynchronously drawn on some interval
function saveCanvas(canvas, destFile) {
return new Promise((resolve, reject) => {
const ext = path.extname(destFile),
encoder = '.png' === ext ? 'pngStream'
: 'jpegStream';
let writable = fs.createWriteStream(destFile),
readable = canvas[encoder]();
writable
.on('finish', resolve)
.on('error', err => {
let msg = `cannot write "${destFile}": ${err.message}`;
reject(new Error(msg));
});
readable
.on('end', () => writable.end())
.on('error', err => {
let msg = `cannot encode "${destFile}": ${err.message}`;
reject(new Error(msg));
});
readable.pipe(writable);
});
}
const FPS = 30;
let frame = 0,
tasks = [],
interval = setInterval(() => tasks.push(
saveCanvas(canvas, `frame_${frame++}.png`)), 1000 / FPS);
// when animation is done, stop timer
// and wait for images to be written
clearInterval(interval);
Promise.all(tasks).then(encodeVideo);
function encodeVideo() {
// too much code to show here, but basically run FFmpeg
// externally with "-i" option containing "frame_%d.png"
// and "-r" = FPS. If you want to encode to VP9 + WEBM,
// definitely see: http://wiki.webmproject.org/ffmpeg/vp9-encoding-guide
}
And then use FFmpeg to encode a sequence of images into a video.
For the code behind encodeVideo(), you can look at this example.
Edit: There may be an issue with canvas.pngStream() writing
incorrect frames while the animation loop continuously draws on
that one canvas--maybe a copy of the canvas needs to be created
per frame? That would surely create significant memory pressure.
I think that the chromium headless mode might support WebGL already and is another possibility. The video rendering part is yet to come though:
https://bugs.chromium.org/p/chromium/issues/detail?id=781117
CCapture.js makes this pretty easy.