I'm writing a bot in Node.js using the MS Bot Framework. To send attachments, I'm actually using the filestream buffer as the contentUrl, e.g.
...
var base64 = new Buffer(filedata).toString('base64');
var msg = new builder.Message()
.setText(session, text)
.addAttachment({
contentUrl: util.format('data:%s;base64,%s', contentType, base64),
contentType: contentType
});
session.send(msg);
...
where contentType is the proper mimetype for the file in question.
When I test this locally (using the Bot Framework Emulator), this works perfectly for both image and audio files - messages with image attachments display the image, and messages with audio attachments show the audiocard allowing for playback, etc.
However, when I test this through FB Messenger, the images work fine, but the audio messages just never appear in FB. Not even the text of the message comes through; it's like the entire message is lost. The dialogue simply skips over the message containing the audio attachment. I'm not even seeing any errors received server-side.
This is happening with both mp3 and wav test audio files, that are each under 1MB (smaller than many of the image files I've successfully tested).
Is there some trick to sending playable audio files to the FB Messenger channel specifically?
Thanks!
I wasn't (yet) able to get a response from FB support, but after further testing, it looks like there is a filesize limit on audio files FB Messenger will accept.
Specifically, I was able to get a sample file of ~45KB to send and display in Messenger successfully, but a larger file of ~400KB got dropped (aka seemed to send successfully from the server-side perspective, but did not show up in Messenger).
Strangely, some of my much larger image files went through, so it seems like this same limit does not exist for image attachments.
Will do some further testing, but it seems like the ultimate solution will be either to majorly compress my audio files, or to host them somewhere else instead of sending as a filestream.
Related
This is the code for messageCreate event that I wrote:
And in the discord app I am getting the response of the console log for no attachments even though I have tried sending images text and audio files:
You're most likely missing the MESSAGE_CONTENT intent.
Go to https://discord.com/developers/applications/ and enable the Message Content Intent in the Bot section.
Image for reference: https://i.imgur.com/mi8XMZb.jpg
Also make sure to add the GatewayIntentBits.MessageContent when you define your client.
I'm trying to make a bot for Facebook Messenger using Microsoft's Bot Framework that will do this:
Get a user's voice message sent via Facebook Messenger
Convert speech to text
Do something with it
There's no problem with getting the voice message from Messenger (the URL can be extracted from the message the bot receives), and there's also no problem with converting an audio file to speech (using Bing Speech API or Google's similar API).
However, these APIs require PCM (WAV) files, while Facebook Messenger gives you an MP4 file.
Is there a popular/standard way of converting one format into another that is used in writing the bots?
So far my best idea is to run vlc.exe as a console job on my server and convert the file, but that doesn't sound like the best solution.
Developed a solution that works as follows:
Receive voice message from facebook
Download the MP4 file to local disk using the link inside Activity.Attachments
Use MediaToolKit (wrapper for FFMPEG) to convert MP4/AAC to WAV on local server
Send the WAV to Bing Speech API
So the answer to my question is: use MediaToolKit+ffmpeg to convert the file format.
Sample implementation and code here: https://github.com/J3QQ4/Facebook-Messenger-Voice-Message-Converter
public string ConvertMP4ToWAV()
{
var inputFile = new MediaFile { Filename = SourceFileNameAndPath };
var outputFile = new MediaFile { Filename = ConvertedFileNameAndPath };
using (var engine = new Engine(GetFFMPEGBinaryPath()))
{
engine.Convert(inputFile, outputFile);
}
return ConvertedFileNameAndPath;
}
I'm trying to write a script that uses the gmail API to forward messages from my inbox to new recipients.
This guide recommends sending the base64 encoded string for the e-mail. Presumably, I could get the existing e-mail using the get method with format=raw, edit the base64 encoding to change the recipients, and send the new message along.
The messages I'm sending are quite large (many attachments), so this process (of downloading a massive base64 string, decoding it, doing some regex substituion, re-encoding it, and then re-uploading it) will take a long time. It also seems very cumbersome to use regex to manipulate a MIME email message.
It seems that there should be an easier way...? Perhaps some way to do this directly via the API?
Try to use the MailApp.sendEmail(). You can use that script to send to multiple receivers so sending messagees to many people wont be that cumbersome.
// Send an email with two attachments: a file from Google Drive (as a PDF) and an HTML file.
var file = DriveApp.getFileById('1234567890abcdefghijklmnopqrstuvwxyz');
var blob = Utilities.newBlob('Insert any HTML content here', 'text/html', 'my_document.html');
MailApp.sendEmail('a#example.com,b#example.com,c#example.com', 'Attachment example', 'Two files are attached.', {
name: 'Automatic Emailer Script',
attachments: [file.getAs(MimeType.PDF), blob]
});
If you want additional samples, try this SO thread.
I am developing a Bot using Microsoft Bot Framework, In that bot will respond with sending some images to the user. I configured it with slack and skype.
In slack Images are displaying but in Skype nothing coming.
To send pictures I used the following syntax
var replyMessage = "![ImgName](" + ImagesUrl + ")";
return message.CreateReplyMessage(replyMessage);
Reference dev.botfrmaework.com, The Text property is Markdown section clearly mention how to link an image to reply message.
If I reply with just link like below, skype able to understand and displaying links. But If I mention like above skype not able to understand.
var replyMessage = "[ImgName](" + ImagesUrl + ")";
return message.CreateReplyMessage(replyMessage);
Each channel has it's own peculiarities. Slack happens to be able to process images sent in-line as you've shown above. However, this will not work generically across all channels. To send images generically, add them as attachments:
replyMessage.Attachments.Add(new Attachment()
{
ContentUrl = "https://upload.wikimedia.org/wikipedia/en/a/a6/Bender_Rodriguez.png",
ContentType = "image/png"
});
Every channel has it's own issues when it comes to images and links.
For instance, the way you send links won't work on FB. it will simply show the links and text.
The same way, the only way to render images in skype as of now is to send it as an attachment and explicitly specify the image format.
Please look this link to see how each platform will react to the markdowns.
And also, for your purpose, I feel adaptive cards will work out more.
Try it out and let me know your comments.
I'm trying to get my receiver to play an mp3 file hosted on the server with the following function
playSound_: function(mp3_file) {
var snd = new Audio("audio/" + mp3_file);
snd.play();
},
However, most of the time it doesn't play and when it does play, it's delayed. When I load the receiver in my local browser, however, it works fine.
What's the correct way to play audio on the receiver?
You can use either a MediaElement tag or Web Audio API. Simplest is probably a MediaElement.