Microsoft Bot Framework attachments for Facebook messenger - node.js

Microsoft Bot Framework messages with buttons in Facebook Messenger
My question relates to the question linked aboved. I am writing a bot using node.js that does not use the bot builder sdk. I manually returning a compatible response for the ms bot connector service. This is working fine for a text response, but I wish to return more complicated responses, e.g the buttons/carousel you can return with messenger. Based on the question I linked above, I guessed the format and added the below:
response.attachments = [ { "Title": "Choose One: ", "Actions": [{ "Title": "Postback!", "Message": "Postback from button" }, { "Title": "Postback2!", "Message": "Postback2 from button" }] } ];
The top level title seems to do nothing but the actions render as postback type buttons correctly (they send the Message as the postback content). With messenger you also have the option to return url based buttons, and image urls.
As far as I can tell there is zero documentation on returning attachments using the node bot builder sdk. If there were I'd just write the bot with the sdk in order to obtain the response format.
So my question is, does anyone know how to correctly return both postback and url based buttons to the bot connnector service, including accompanying images, with or without the bot builder sdk?
Update 05/05/2016
So I found the link below and you can see a definition of the attachments property:
http://docs.botframework.com/sdkreference/nodejs/interfaces/_botbuilder_d_.imessage.html
If you follow it to the IAttachment specification, it makes me wonder how/why my code above works at all? As a test of that format I wrote in the following:
var att = {};
att.content = "I am content";
att.contentType = "text/plain";
att.contentUrl = "http://www.google.com";
att.fallbackText = "I am fallback text";
att.text = "I am text";
att.thumbnailUrl = "https://pbs.twimg.com/profile_images/638751551457103872/KN-NzuRl.png";
att.title ="I am title";
att.titleLink = "http://yahoo.com";
Now in slack I get a fairly nice output from this:
However in messenger I get "Service Error:Value cannot be null. Parameter name: source"

I found the info I needed. Not sure if it hadn't been published at the time or whether I was just hunting in the bot builder docs, but it's all detailed fairly well below.
http://docs.botframework.com/connector/message-actions/#navtitle
You have to tweak your message a little for certain integrations, e.g Skype doesn't really seem to support attachments.

Related

How to send rich text with image using Microsoft Bot Framework

Recently we are building a application which send message through Microsoft BotFramework API
We are using the API as below
https://learn.microsoft.com/en-us/azure/bot-service/rest-api/bot-framework-rest-connector-api-reference?view=azure-bot-service-4.0#send-to-conversation
The following API allow us to send text or attachments by passing activities as parameters.
https://learn.microsoft.com/en-us/azure/bot-service/rest-api/bot-framework-rest-connector-api-reference?view=azure-bot-service-4.0#activity-object
Everything works well, and on our client side, text and attachments can be received normally as below.
However, recently we have to send rich text which contains text and image info. For example, admin user want to send text and pasted image together to client user.
So we are considering sending text with markdown style with image info like this
text:aaaaa![test](https://ichef.bbci.co.uk/news/976/cpsprodpb/C448/production/_117684205_lotus.jpg "test")
however, client user can only get the info as below. On our client side who use Microsoft Teams App, the image info could not be parsed normally, although image src is a public link.
I know by using attachment image can be send successfully, but what we need is to send a rich text which has ordered text and image as below
Could someone tell me how to find a solution?
I have tested the given markdown syntax in MS Team and it's rendering the correct output. Here through QnA maker I have verified & tested the flow instead of the API that you have given. The issue is not related to Teams APP and look like the markdown has one extra text in the url probably that creating the problem while posting attachment in the MS Teams through request body.
Your Markdown Syntax
Remove the extra "test" content from url.
text:aaaaa![test](https://ichef.bbci.co.uk/news/976/cpsprodpb/C448/production/_117684205_lotus.jpg "test")
Markdown Syntax
text:aaaaa![test](https://ichef.bbci.co.uk/news/976/cpsprodpb/C448/production/_117684205_lotus.jpg)
Output
Note: QnA maker will automatically remove this kind of extra content from the url during the build & test process.
This is possible using Language Generation file in your project and using Adaptive cards. You can find samples here https://adaptivecards.io/samples/
You can design your own adaptive card in your designer https://adaptivecards.io/designer/
and create a json file which you will have to place inside your project Template folder. (or respective folder directory that you have registered)
Once you are done creating and placing the files, follow the code to call the adaptive card
await dc.Context.SendActivityAsync(ActivityFactory.FromObject(_templates.Evaluate("yourlgintent"))).ConfigureAwait(false);
in your filename.lg file
# yourlgintent
[Activity
Attachments = ${ActivityAttachment(json(fromFile('yourfilename.json')), 'adaptiveCard')}
]
OR
You can directly call it in a method
# yourlgintent
[Activity
Attachments = ${ActivityAttachment(json(yourlgintentJson()), 'adaptiveCard')}
]
#yourlgintentJson
- ```
{
"$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
"type": "AdaptiveCard",
"version": "1.3",
"body": [
{
"type": "Container",
"items": [
{
"type": "TextBlock",
"wrap": true,
"text": "As your virtual assistant, I love answering your questions. Having said that, I am learning continuously to expand my knowledge.",
"height": "stretch"
}
],
"spacing": "Large"
}
]
}
And yes, If you are looking to get image from your QnA, then follow the below code
Inside your QnAMaker portal ,
Your Answer. [Image](https://URLTotheimage.com/Images/image.png)
This should give image within your answer.
Finally I found the sollution
https://github.com/microsoft/AdaptiveCards/issues/4121

How can users upload images to a Slack app?

To provide context i have a Slack bot that allows users to create ads, i am able to use a dialog to fetch the listing title, description and price. What am looking for is a way to allow users to also add images.
The file.upload seems to allow the bot to upload files but what i want is lo be able to allow users to select the files locally and upload them, the bot will then be able to capture this and respond accordingly.
This is what i have so far
#app.route('/new', methods=['POST'])
def new_listing():
# Get payload
api_url = 'https://slack.com/api/dialog.open'
trigger_id = request.form.get('trigger_id')
dialog = {
"callback_id": "marketplace",
"title": "Create a new listing",
"submit_label": "Create",
"notify_on_cancel": True,
"state": "Item",
"elements": [
{
"type": "text",
"label": "Listing Title",
"name": "listing_title"
},
{
"type": "text",
"label": "Listing description",
"name": "listing_description"
},
{
"type": "text",
"label": "Listing Price",
"name": "listing_price"
}
]
}
api_data = {
"token": oauth_token,
"trigger_id": trigger_id,
"dialog": json.dumps(dialog)
}
res = requests.post(api_url, data=api_data)
print(res.content)
return make_response()
#app.route('/message_actions', methods=['POST'])
def message_actions():
user_id = request.form['user']['id']
submission = request.form['submission']
title = submission['listing_title']
description = submission['listing_description']
price = submission['listing_price']
# Add the listing to the database
return make_response()
There is no straight forward approach, since the Slack API (currently) does not offer a filer picker.
However, here are 3 workarounds to address this requirement:
A - Image URLs
Instead of uploading images to Slack directly, users only provide the URL of image hosted on the Internet (e.g. uploading to imgur.com). The image URL can be queried with a simple plain-text input field in your dialog.
If you can expect your users to be tech savvy enough to handle image URLs and uploads to imgur.com (or other image hosters) I think think approach works pretty well.
B - External web page
You redirect users to an external web page of your app that has a file picker. That file picker allows uploading images from the user local machine to your app.
This approach also works well. However users need to switch to a browser (and back to Slack again), so it can break the input flow a bit. It also is a lot more effort to implement, e.g. you need to maintain context between Slack and your web page in a secure way, which can be a challenge.
C - Manual upload to Slack
Users upload images manual to Slack, e.g. in the app channel. You app detects every image upload and asks them to which item of your app to attach it to.
This approach allows you to stay within the Slack eco-system, but might be confusing for users and ensuring correct linking between user uploads and your items might be a challenge.
P.S.: I had the same requirement with one of my Slack apps (Rafflebot) and went with approach A.
You don't show how you are invoking /new (with the trigger id). However - while dialogues and the new modals don't seem to have file pickers - the slack app certainly does. So what I do is start my flow off with a message to my app - THAT message can have files attached. So for example my app looks that the message 'new report' - the user, prior to sending that can attach images - and my app will both get the message AND get a "files" attributes as part of the message event.

Google Dialogflow Quick Replies formatting issues in Telegram

I had create a chatbot using Dialogflow and integrated it with Facebook Messenger & Telegram. I noticed that for the Quick Replies in Telegram (Link 1) appears differently in FB Messenger (Link 2). Is there any way to make it nicer and more presentable in Telegram?
Telegram
Facebook Messenger
This is my Quick Replies settings in Dialogflow.
Dialogflow
in DialogFlow you can indeed (as Marc pointed out) use a Custom Payload for Telegram, here it is an example:
{
"telegram": {
"text": "What would you like help with?",
"reply_markup": {
"inline_keyboard": [
[
{
"text": "Daily News",
"callback_data": "news"
}
],
[
{
"text": "New Features",
"callback_data": "features"
}
]
]
}
}
}
The quick replies appear a buttons you can click (notice the actual response is sent but not displayed within the chat).
All the best!
Beppe
SHORT ANSWER:
That is NOT possible.
DETAILED ANSWER:
Each Channel (Facebook Messenger, Telegram etc.) has its own UI components and its own styling. Those can't be changed as they are rendered/controlled by the channel it self.
What Dialogflow doing is giving you the ability to show these UI components in each channel without making you handle the different implementation needed for each channel.
Dialogflow also gives you the ability to send Custom Payloads where you can send a custom JSON (that should be compatible with the channel you are connected to) That can be used if the channel for example has a UI component that is not yet supported by Dialogflow.
If the channel gives you an option to change a property in the UI components you are using, you could do that using Custom JSON, But still you are always limited to how each channel renders the UI components and what features they provide us

DialogFlow V2 How set fulfillmentText via node js library?

Following this guide:
https://actions-on-google.github.io/actions-on-google-nodejs/
I created an action for DialogFlow
import { dialogflow, Image, Conversation, BasicCard } from 'actions-on-google';
const app = dialogflow();
app.intent('test', (conv, input) => {
conv.contexts.set('i_see_context_in_web_demo', 1);
conv.ask(`i see this only into actions on google simulator`);
conv.ask(new Image({
url: 'https://developers.google.com/web/fundamentals/accessibility/semantics-builtin/imgs/160204193356-01-cat-500.jpg',
alt: 'cat',
}));
});
I then activated Web Demo integration
I saw that the Web Demo integration does not show the cards, the images. I hypothesize that it only shows text, no rich text
I understand that it elaborates only JSON like this:
{
"fulfillmentText": "Welcome!",
"outputContexts": []
}
But I did not find any method in the library used to enhance fulfillmentText
can you help me?
You're using the actions-on-google library, which is specifically designed to send messages that will be used by the Google Assistant. The Web Demo uses the generic messages that are available for Dialogflow. The actions-on-google library does not send these generic messages.
If you want to be able to create messages that are usable by both, you'll need to look into the dialogflow fulfillment library, which can create messages that are usable by the Google Assistant as well as other platforms. Be aware, however, that not all rich messages are available on all platforms, but that the basic text responses should be.
You also don't need to use a library - you can create the JSON response yourself.

How to display images in Skype Bot with Microsoft Bot as Client

I am developing a Bot using Microsoft Bot Framework, In that bot will respond with sending some images to the user. I configured it with slack and skype.
In slack Images are displaying but in Skype nothing coming.
To send pictures I used the following syntax
var replyMessage = "![ImgName](" + ImagesUrl + ")";
return message.CreateReplyMessage(replyMessage);
Reference dev.botfrmaework.com, The Text property is Markdown section clearly mention how to link an image to reply message.
If I reply with just link like below, skype able to understand and displaying links. But If I mention like above skype not able to understand.
var replyMessage = "[ImgName](" + ImagesUrl + ")";
return message.CreateReplyMessage(replyMessage);
Each channel has it's own peculiarities. Slack happens to be able to process images sent in-line as you've shown above. However, this will not work generically across all channels. To send images generically, add them as attachments:
replyMessage.Attachments.Add(new Attachment()
{
ContentUrl = "https://upload.wikimedia.org/wikipedia/en/a/a6/Bender_Rodriguez.png",
ContentType = "image/png"
});
Every channel has it's own issues when it comes to images and links.
For instance, the way you send links won't work on FB. it will simply show the links and text.
The same way, the only way to render images in skype as of now is to send it as an attachment and explicitly specify the image format.
Please look this link to see how each platform will react to the markdowns.
And also, for your purpose, I feel adaptive cards will work out more.
Try it out and let me know your comments.

Resources