Dialogflow v2 API - cards not shown in the simulator - dialogflow-es

I've a webhook for fulfillment.
Below is the code that's responding back
let result_obj = {
"fulfillmentText": "This is a text response",
"fulfillmentMessages": [
{
"text": {
"text": [
"this is test"
]
}
},
{
"card": {
"title": "card title",
"subtitle": "card text",
"imageUri": "https://assistant.google.com/static/images/molecule/Molecule-Formation-stop.png",
"buttons": [
{
"text": "button text",
"postback": "https://assistant.google.com/"
}
]
}
}
]
}
Below is the result from dialogflow GUI
Below is what I get when I run from the simulator or from the Google Assistant application on the Android phone
Both the simulator and phone are not showing the cards. Am I missing something obvious here?

For rich responses like cards to show on Google Assistant you have to use the payload part of response JSON, here is an example:
{
"fulfillmentText": "This is a text response",
"fulfillmentMessages": [],
"source": "example.com",
"payload": {
"google": {
"expectUserResponse": true,
"richResponse": {
"items": [
{
"simpleResponse": {
"textToSpeech": "This is a Basic Card:"
}
},
{
"basicCard": {
"title": "card title",
"image": {
"url": "https://www.google.com/images/branding/googlelogo/2x/googlelogo_color_272x92dp.png",
"accessibilityText": "Google Logo"
},
"buttons": [
{
"title": "Button Title",
"openUrlAction": {
"url": "https://www.google.com"
}
}
],
"imageDisplayOptions": "WHITE"
}
}
]
}
}
},
"outputContexts": [],
"followupEventInput": {}
}
Check out this github repo for all rich-responses' JSON formats.

Related

How To Transcrib Audio To Text In Dialogflow-ES

I have a chatbot in dialogflow, it is conected with whatsapp by landbot, in this moment when an user send me an audio, landbot send the audio to dialogflow(literal in the format .ogv) but I need to transcrib it, because the dialogflow must understand. For example, this is the intent:
{
"id": "ffdc721f-8e94-4a9e-9ec8-a45b9b7f21be-37284719",
"fulfillmentText": "🙁 Pronunciaste mal. \n La palabra era: Dog",
"language_code": "en",
"queryText": "https://media.eu-1.smooch.io/apps/5d2370ef6667cd00102fb9c2/conversations/31f2d6d5440d03fde4066b35/hJ9ob-c2Ogho4NTDLEuyFMg_/5zKf3-SgGMuS3RxcBUw5B6dj.oga",
"webhookPayload": {},
"intentDetectionConfidence": 0.3,
"action": "",
"webhookSource": "",
"parameters": {
"pronunciacion": "https://media.eu-1.smooch.io/apps/5d2370ef6667cd00102fb9c2/conversations/31f2d6d5440d03fde4066b35/hJ9ob-c2Ogho4NTDLEuyFMg_/5zKf3-SgGMuS3RxcBUw5B6dj.oga",
"palabra": "Dog"
},
"fulfillmentMessages": [
{
"text": {
"text": [
"🙁 Pronunciaste mal. \n La palabra era: Dog"
]
}
}
],
"diagnosticInfo": {
"webhook_latency_ms": "1871.0"
},
"webhookStatus": {
"webhookStatus": {
"message": "Webhook execution successful"
},
"webhookUsed": true
},
"intent": {
"isFallback": false,
"displayName": "Pronunciar",
"id": "4dd12af2-94a6-486b-a1cd-daa2c65d6671"
}
}
I searched that I can do that with Speech-To-Text but I don't know how can I put it in dialogflow

Unable to post message via the Slack API - getting error 'no_text'

Why am I getting an error 'no_text' ?
The Json below is valid and is taken from the slack documentation example.
In bot.postMessage(channel, '', params) if I populate the second parameter (i.e. replacing ' ' with 'some_text) it prints 'some_text' but without the attachment.
bot.postMessage(channel, 'some_text', params) --> works but the attachment doesn't show up.
const element = {
"text": "Would you like to play a game?",
"response_type": "in_channel",
"attachments": [
{
"text": "Choose a game to play",
"fallback": "If you could read this message, you'd be choosing something fun to do right now.",
"color": "#3AA3E3",
"attachment_type": "default",
"callback_id": "game_selection",
"actions": [
{
"name": "games_list",
"text": "Pick a game...",
"type": "select",
"options": [
{
"text": "Hearts",
"value": "hearts"
},
{
"text": "Global Thermonuclear War",
"value": "war"
}
]
}
]
}
]
}
console.log('JSON.stringify(element): '+JSON.stringify(element));
params = {
icon_emoji: ':r2:',
attachments: JSON.stringify(element)
}
bot.postMessage(channel, '', params).always(function (data) {...}
The issue arises from the lack of a text field in the parameters that is passed to bot.PostMessage. Your params should be
params = {
icon_emoji: ':r2:',
text: "Would you like to play a game?",
response_type: "in_channel",
attachments: element
}
and the element now should start from the actual attachment
const element = [
{
"text": "Choose a game to play",
"fallback": "If you could read this message, you'd be choosing something fun to do right now.",
"color": "#3AA3E3",
"attachment_type": "default",
"callback_id": "game_selection",
"actions": [
{
"name": "games_list",
"text": "Pick a game...",
"type": "select",
"options": [
{
"text": "Hearts",
"value": "hearts"
},
{
"text": "Global Thermonuclear War",
"value": "war"
}
]
}
]
}
]
I met the same issue and found the solution.
The problem is that if only attachments field is added into the payload, it will report no_text error. But if text field is added, slack message will only show the text content.
The solution:
When we want to display attachments, we need to add a basic blocks field instead of text field. Something like
{
"blocks": [
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "bar"
}
}
],
"attachments": [
{
"color": "#FF0000",
"blocks": [
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "foo"
}
}
]
}
]
}
If putting the above payload into the Slack build kit, it will be a misleading. That's also why I stuck with the issue.
I would recommend to use chat.postMessage test to debug the payload. It will work like a charm.

How to make a carousel using Google.Cloud.Dialogflow.V2 Carousel for dialogflow?

Im trying to build a webhook for dialogflow with help from nuget package Google.Cloud.Dialogflow.V2.
Im running the code with .Net Core 2.1.
I get the fullfilmentText to show in dialogflow, but cant make the carousel to show up.
What am I doing wrong?
Here is my method:
[HttpPost]
public ContentResult Post([FromBody] WebhookRequest webhookRequest)
{
var dialogflowResponse = new WebhookResponse { FulfillmentText = "FulfillmentText" };
var carousel = new Intent.Types.Message.Types.CarouselSelect.Types.Item
{
Title = "My title",
Description = "This is a description",
Image = new Intent.Types.Message.Types.Image
{
ImageUri = "https://placekitten.com/200/300",
AccessibilityText = "cat"
},
Info = new Intent.Types.Message.Types.SelectItemInfo
{
Key = "Cats"
}
};
carousel.Info.Synonyms.Add("Synonym 1");
carousel.Info.Synonyms.Add("Synonym 1");
dialogflowResponse.FulfillmentMessages.Add(new Intent.Types.Message {
CarouselSelect = new Intent.Types.Message.Types.CarouselSelect()
});
dialogflowResponse.FulfillmentMessages.First().CarouselSelect.Items.Add(carousel);
return dialogflowResponse;
}
Generates followiing Json:
{
"fulfillmentText": "FulfillmentText",
"fulfillmentMessages": [
{
"carouselSelect": {
"items": [
{
"info": {
"key": "Cats",
"synonyms": [
"Synonym 1",
"Synonym 1"
]
},
"title": "My title",
"description": "This is a description",
"image": {
"imageUri": "https://placekitten.com/200/300",
"accessibilityText": "cat"
}
}
]
}
}
]
}
i ran into the same problem with the java api. Only the basic rich messages (card, quick replies, ...) are working. BasicCard, Suggestions, Carousel, ... doesn't show up in any channel.
I got it to work if i added some of the following custom payload to skype channel:
{
"skype": {
"type": "message",
"attachmentLayout": "carousel",
"text": "Please select your food.",
"attachments": [{
"contentType": "application/vnd.microsoft.card.hero",
"content": {
"title": "Sushi",
"subtitle": "Very fresh shrimp or tuna.",
"images": [{
"url": "https://i.ibb.co/87LmvT3/mdi.png"
}
],
"buttons": [{
"type": "imBack",
"title": "1 piece",
"value": "sushi,1"
}, {
"type": "imBack",
"title": "2 piece",
"value": "sushi,2"
}, {
"type": "imBack",
"title": "more",
"value": "sushi,3+"
}
]
}
}, {
"contentType": "application/vnd.microsoft.card.hero",
"content": {
"title": "Tenpura",
"subtitle": "Japanese first-class vegitables.",
"images": [{
"url": "https://i.ibb.co/87LmvT3/mdi.png"
}
],
"buttons": [{
"type": "imBack",
"title": "1 piece",
"value": "tenpura,1"
}, {
"type": "imBack",
"title": "2 piece",
"value": "tenpura,2"
}, {
"type": "imBack",
"title": "more",
"value": "tenpura,3+"
}
]
}
}
]
}}
I hope it helps!

webhook doesn't receive event from m.me link

I've been trying to pass parameter using m.me link, but when tester/ developer account of my app tried this feature, my webhook didn't receive any event.
My bot already has a get started button and the page is published.
what im trying to ask is why couldn't i get any event and how to solve this problem?
let message = {
"attachment": {
"type": "template",
"payload": {
"template_type": "generic",
"elements": [
{
"title": "Share me 😊",
"subtitle": "share to your friend",
"image_url": "https://gallery.yopriceville.com/var/resizes/Free-Clipart-Pictures/Sun-and-Moon-PNG/Transparent_Sun_with_Shades_PNG_Clipart_Picture.png?m=1434276589",
"buttons": [
{
"type": "element_share",
"share_contents": {
"attachment": {
"type": "template",
"payload": {
"template_type": "generic",
"elements": [
{
"title": "Add to connect to your friend",
"subtitle": "connect",
"image_url": output.profile_pic,
"default_action": {
"type": "web_url",
"url": `http://m.me/${page_id}?ref=invited_by_${recipientId}`
},
"buttons": [
{
"type": "web_url",
"url": `http://m.me/${page_id}?ref=invited_by_${recipientId}`,
"title": `ADD ${output.first_name}`
}
]
}
]
}
}
}
}
]
}
]
}
}
};

How to determine the type of the platform in Dialogflow (API.AI)

How do to identify from which platform the message came?
I want to support different platforms like Telegram and Facebook Messenger, When my webhook receive a message, I want to reply according to the platform the message came form.
For example, if the message came from Telegram I want to return a text message but if the message came from messenger I want to return a card.
You have a property source in originalRequest object, see fulfillment docs here.
{
"lang": "en",
"status": {
"errorType": "success",
"code": 200
},
"timestamp": "2017-02-09T16:06:01.908Z",
"sessionId": "1486656220806",
"result": {
"parameters": {
"city": "Rome",
"name": "Ana"
},
"contexts": [],
"resolvedQuery": "my name is Ana and I live in Rome",
"source": "agent",
"score": 1.0,
"speech": "",
"fulfillment": {
"messages": [
{
"speech": "Hi Ana! Nice to meet you!",
"type": 0
}
],
"speech": "Hi Ana! Nice to meet you!"
},
"actionIncomplete": false,
"action": "greetings",
"metadata": {
"intentId": "9f41ef7c-82fa-42a7-9a30-49a93e2c14d0",
"webhookForSlotFillingUsed": "false",
"intentName": "greetings",
"webhookUsed": "true"
}
},
"id": "ab30d214-f4bb-4cdd-ae36-31caac7a6693",
"originalRequest": {
"source": "google",
"data": {
"inputs": [
{
"raw_inputs": [
{
"query": "my name is Ana and I live in Rome",
"input_type": 2
}
],
"intent": "assistant.intent.action.TEXT",
"arguments": [
{
"text_value": "my name is Ana and I live in Rome",
"raw_text": "my name is Ana and I live in Rome",
"name": "text"
}
]
}
],
"user": {
"user_id": "PuQndWs1OMjUYwVJMYqwJv0/KT8satJHAUQGiGPDQ7A="
},
"conversation": {
"conversation_id": "1486656220806",
"type": 2,
"conversation_token": "[]"
}
}
} }

Resources