I am trying to record google meet audio and video using getDisplayMedia using media options.I am able to record my screen through getDisplayMedia but unable to record voice coming in google meet. How can I use passive recording or is there any way where I can record screen, audio and video in google meet through any mechanism or code?
navigator.mediaDevices.getDisplayMedia({audio: true,video: true});
let mediaRecorder = new MediaRecorder(mediaStreamObj);
let chunks = [];
mediaRecorder.start();
console.log(mediaRecorder.state);
mediaRecorder.ondataavailable = function(ev) {
chunks.push(ev.data);
}
setTimeout(function() {
console.log("Stopping!");
mediaRecorder.stop();
console.log(mediaRecorder.state);
mediaRecorder.onstop = (ev) => {
let blob = new Blob(chunks, {
'type': 'video/mp4;'
});
chunks = [];
let videoURL = window.URL.createObjectURL(blob);
console.log(videoURL);
}
}, 40000);
i did it with puppeteer. i build a google meet bot who will join your meetings from google calendar and record audio + save all transcript . when meeting ends everything will be sent over email or you can see in app dashboard.
Related
Few weeks ago I wrote WebRTC browser client using mediasoup library. Now I am in middle of rewriting it as a NodeJS client.
I am stuck with one thing. I want to receive multiple WebRTC audio souroces, mix them into single track then apply some filters(eg. biquad filter) and then resend this track via WebRTC. Like on this diagram:
With browser I could achieve this using Web Audio API this is the code I used:
this.audioContext = new AudioContext();
this.outgoingStream = this.audioContext.createMediaStreamDestination();
this.addSoundFilter();
this.mixedTrack = this.outgoingStream.stream.getAudioTracks()[0];
this.handleIncomingSound();
addSoundFilter() {
this.filter = this.audioContext.createBiquadFilter();
this.filter.type = "lowpass";
this.filter.frequency.value = this.mapFrequencyValue();
this.gainer = this.audioContext.createGain();
this.gainer.gain.value = this.mapGainValue();
}
handleIncomingSound() {
this.audios.forEach((audio, peerId) => {
this.filterAudio(audio);
});
}
filterAudio(audioConsumerId) {
let audio = this.audios.get(audioConsumerId);
const audioElement = document.getElementById(audio.id);
const incomingSource = this.audioContext.createMediaStreamSource(
audioElement.srcObject
);
incomingSource.connect(this.filter).connect(this.gainer).connect(this.outgoingStream);
}
And with this code I could then send this.mixedTrack via WebRTC
However in NodeJS there is no WebAudioApi.
So how this can be achieved or if it's even possible to do it?
I am going to develop a simple web based Speech to Text project. Develop with NodeJS, ws (WebSocket), and Google's Speech to Text API.
However, I have no luck to get the transcript from Google's Speech to Text API.
Below are my server side codes (server.js):
ws.on('message', function (message) {
if (typeof message === 'string') {
if(message == "connected") {
console.log(`Web browser connected postback.`);
}
}
else {
if (recognizeStream !== null) {
const buffer = new Int16Array(message, 0, Math.floor(message.byteLength / 2));
recognizeStream.write(buffer);
}
}
});
Below are my client side codes (ws.js):
function recorderProcess(e) {
var floatSamples = e.inputBuffer.getChannelData(0);
const ConversionFactor = 2 ** (16 - 1) - 1;
var floatSamples16 = Int16Array.from(floatSamples.map(n => n * ConversionFactor));
ws.send(floatSamples16);
}
function successCallback(stream) {
window.stream = stream;
var audioContext = window.AudioContext;
var context = new audioContext();
var audioInput = context.createMediaStreamSource(stream);
var recorder = context.createScriptProcessor(2048, 1, 1);
recorder.onaudioprocess = recorderProcess;
audioInput.connect(recorder);
recorder.connect(context.destination);
}
When I run the project, and open http://localhost/ in my browser, trying to speaking some sentences to the microphone. Unfortunately, there are no transcription returned, and no error messages returned in NodeJS console.
When I check the status in Google Cloud Console, it only display a 499 code in the dashboard.
Many thanks for helping!
I think the issue could be related to the stream process. Maybe some streaming process is stopped before the end of an operation. My suggestion is to review the callbacks in the JasvaScript code in order to find some “broken" promises.
Also, maybe its obvious but there is a different doc for audios than more than a minute:
https://cloud.google.com/speech-to-text/docs/async-recognize
CANCELLED - The operation was cancelled, typically by the caller.
HTTP Mapping: 499 Client Closed Request
Since the error message, this also could be related to the asynchronous and multithread features of node js.
Hope this works!
I want to send a card to a teams channel using messaging extension. On messaging extension i need to show a preview thumbnail card and onclick of that thumbnail a adaptive card will be displayed.
I have tried the below code and while trying to use "MessagingExtensionResult" its giving error. Also i'm unable to add the dll for "MessagingExtensionResult" its giving incompatible version error. I'm using .Net framework 4.6.
var results = new ComposeExtensionResult()
{
AttachmentLayout = "list",
Type = "result",
Attachments = new List<ComposeExtensionAttachment>(),
};
var card = CardHelper.CreateCardForExperties(pos, true);
var composeExtensionAttachment = card.ToAttachment().ToComposeExtensionAttachment();
results.Attachments.Add(new ComposeExtensionAttachment
{
ContentType = "application/vnd.microsoft.teams.card.adaptive",
Content = JsonConvert.DeserializeObject(updatedJsonString),
Preview = composeExtensionAttachment
});
Using below code we can invoke adaptive card from thumbnail card preview.
ComposeExtensionResponse response = null;
1. var results = new ComposeExtensionResult()
{
AttachmentLayout = "list",
Type = "result",
Attachments = new List<ComposeExtensionAttachment>(),
};
Create a function that returns thumbnail card (preview card)
var previewThumbnailCard = CreateThumbnailCard();
Create a function that returns Adaptive card in form of attachment.
var adaptivecardattachment = CreateAdaptiveCardAsAttachment();
Cast that attachment card to composeextensionattachment and pass
thumbnail card to it as attachment.
var composeExtensionAttachmentAdaptive = adaptivecardattachment .ToComposeExtensionAttachment(previewThumbnailCard.ToAttachment());
Return the response
{
ComposeExtension = results
};
return response;
I am implementing a cisco spark bot , which is now known as Webex teams. I am able to send receive simple text message from user. Is there a way to send rich card to user from Bot? I could not find any helpful documentation.
Here is my code I use to interact with bot
var SparkBot = require("node-sparkbot");
var SparkAPIWrapper = require("node-sparkclient");
// Starts your Webhook with default configuration where the SPARK API access
token is read from the SPARK_TOKEN env variable
var bot = new SparkBot();
var spark = new SparkAPIWrapper(process.env.SPARK_TOKEN);
bot.onMessage(function (trigger, message) {
if (message.personEmail != "mytestbot#webex.bot")
spark.createMessage(message.roomId, "You said " + message.text, {
"markdown":
true }, function (err, message) {
if (err) {
console.log("WARNING: could not post message to room: " +
message.roomId);
return;
}
});
});
Rich cards may contain buttons , links, list , thumbnail etc..
This is not supported yet on the platform side. The best you have at the moment is you can use markdown to style messages.
How to create google chrome extension with a notification for every change in my website page ?
If your site has RSS feed, you can do something like this (only new pages):
Get and parse RSS feed using jQuery:
$.get('http://yoursite.com/rss', function(data) {
$(data).find('item').each(function() {
var url_news = $(this).find('link').text();
var news_id = $(this).find('id').text();
var descr = $(this).find('title').text();
});
});
Show Notify using Chrome Notification
https://developer.chrome.com/extensions/notifications
var title = "Title";
var notification = webkitNotifications.createNotification(img, title, descr);
notification.onclick = function() {
chrome.tabs.create({url: url_news}, notification.cancel());
chrome.windows.getLastFocused(null, function(win){
if(win.state=="minimized"){
chrome.windows.update(win.id, {state:"normal", focused:true});
}
});
};
notification.show();
When updated your rss feed, chrome extension will display notifications