I have chrome extension (written in Manifest v2) that uses MediaRecorder to record the web session
In background.js I have below code to capture web session
chrome.tabs.captureVisibleTab(null, {format: 'png', quality: 100},
function(base64) {
// base64 is used to write to the canvas
// and used canvas.getMediaStream() to get the stream
// and stream passed as input to mediaRecorder
});
now I have to migrate to Manifest v3 where service worker don't have access to mediaRecorder and I am looking for alternative to it.
I have also tried to send base64 to the content script to write it onto canvas and to use canvas stream and media recorder available on the content script, but as and when page refreshes stream ends and new canvas and stream get created which causes video played in different tracks.
please let me know if anyone has solution for it
Related
I'm writing a bot in Node.js using the MS Bot Framework. To send attachments, I'm actually using the filestream buffer as the contentUrl, e.g.
...
var base64 = new Buffer(filedata).toString('base64');
var msg = new builder.Message()
.setText(session, text)
.addAttachment({
contentUrl: util.format('data:%s;base64,%s', contentType, base64),
contentType: contentType
});
session.send(msg);
...
where contentType is the proper mimetype for the file in question.
When I test this locally (using the Bot Framework Emulator), this works perfectly for both image and audio files - messages with image attachments display the image, and messages with audio attachments show the audiocard allowing for playback, etc.
However, when I test this through FB Messenger, the images work fine, but the audio messages just never appear in FB. Not even the text of the message comes through; it's like the entire message is lost. The dialogue simply skips over the message containing the audio attachment. I'm not even seeing any errors received server-side.
This is happening with both mp3 and wav test audio files, that are each under 1MB (smaller than many of the image files I've successfully tested).
Is there some trick to sending playable audio files to the FB Messenger channel specifically?
Thanks!
I wasn't (yet) able to get a response from FB support, but after further testing, it looks like there is a filesize limit on audio files FB Messenger will accept.
Specifically, I was able to get a sample file of ~45KB to send and display in Messenger successfully, but a larger file of ~400KB got dropped (aka seemed to send successfully from the server-side perspective, but did not show up in Messenger).
Strangely, some of my much larger image files went through, so it seems like this same limit does not exist for image attachments.
Will do some further testing, but it seems like the ultimate solution will be either to majorly compress my audio files, or to host them somewhere else instead of sending as a filestream.
I'm trying to make a bot for Facebook Messenger using Microsoft's Bot Framework that will do this:
Get a user's voice message sent via Facebook Messenger
Convert speech to text
Do something with it
There's no problem with getting the voice message from Messenger (the URL can be extracted from the message the bot receives), and there's also no problem with converting an audio file to speech (using Bing Speech API or Google's similar API).
However, these APIs require PCM (WAV) files, while Facebook Messenger gives you an MP4 file.
Is there a popular/standard way of converting one format into another that is used in writing the bots?
So far my best idea is to run vlc.exe as a console job on my server and convert the file, but that doesn't sound like the best solution.
Developed a solution that works as follows:
Receive voice message from facebook
Download the MP4 file to local disk using the link inside Activity.Attachments
Use MediaToolKit (wrapper for FFMPEG) to convert MP4/AAC to WAV on local server
Send the WAV to Bing Speech API
So the answer to my question is: use MediaToolKit+ffmpeg to convert the file format.
Sample implementation and code here: https://github.com/J3QQ4/Facebook-Messenger-Voice-Message-Converter
public string ConvertMP4ToWAV()
{
var inputFile = new MediaFile { Filename = SourceFileNameAndPath };
var outputFile = new MediaFile { Filename = ConvertedFileNameAndPath };
using (var engine = new Engine(GetFFMPEGBinaryPath()))
{
engine.Convert(inputFile, outputFile);
}
return ConvertedFileNameAndPath;
}
I have a Chrome extension, which detects Youtube videos and gets their category via the Youtube API as follows:
background.js
chrome.webRequest.onBeforeRequest.addListener(function(details) {
var PageInfo = new URL(url);
if(PageInfo.host=="www.youtube.com")
getCategory(PageInfo); //returns category number via the Youtube API
});
The problem I'm facing is that this does not work if Youtube is in full screen mode. For example after I load a video normally, I change to full screen and pick a video from the suggested videos after the initial one has finished playing. I then cannot get the extension to correctly pick up that new video.
Any tips would be appreciated. Thanks.
I'm trying to get my receiver to play an mp3 file hosted on the server with the following function
playSound_: function(mp3_file) {
var snd = new Audio("audio/" + mp3_file);
snd.play();
},
However, most of the time it doesn't play and when it does play, it's delayed. When I load the receiver in my local browser, however, it works fine.
What's the correct way to play audio on the receiver?
You can use either a MediaElement tag or Web Audio API. Simplest is probably a MediaElement.
We want to play Smoothstreaming URL by providing Manifest file of smoothstreaming to Chromecast device.
We could play the following on Chromecast device,
1. .mp4 file
2. .ismv file
3. .isma file.
But, if we provide an Manifest file as follows, we are not able to play on Chromecast device.
http://playready.directtaps.net/smoothstreaming/SSWSS720H264/SuperSpeedway_720.ism/Manifest
Please let us know how to play Smoothstreaming URL on Chromecast device.
Or do we need to play .ismv files one by one by providing them in a loop.
The Chromecast has support for SmoothStreaming content through their Media Player Library: https://developers.google.com/cast/docs/player
Below is a bare bones implementation.
Google provides a proper example on GitHub which takes advantage of the MediaManager and accounts for other streaming formats: https://github.com/googlecast/CastMediaPlayerStreamingDRM)
var $mediaElement = $('<video>').attr('autoplay', ''),
mediaElement = $mediaElement[0],
mediaUrl = "http://playready.directtaps.net/smoothstreaming/SSWSS720H264/SuperSpeedway_720.ism/Manifest",
mediaHost,
mediaPlayer;
cast.receiver.CastReceiverManager.getInstance().start();
$('body').append(mediaElement);
mediaHost = new cast.player.api.Host({
mediaElement: mediaElement,
url: mediaUrl
});
var protocol = cast.player.api.CreateSmoothStreamingProtocol(mediaHost);
mediaPlayer = new cast.player.api.Player(mediaHost);
mediaPlayer.load(protocol);
Microsoft's test files (inc the ISM) do not return the CORS header required for the chromecast. use a CORS on all of your server and it shall work.
I've encountered this too, and its working if I host them myself with CORS