Dialogflow BrowseCarousel not working after Permission granted - dialogflow-es

In my flow there are three handlers:
Store finder: when user ask "where is store near me?", it triggers the actions.intent.PERMISSION that ask user for his precise location.
Store finder - yes: if user replies "yes", this will be triggered and the nearest stores will be shown (based on lat/long extracted from request)
Store finder - no: if user replies "no", this will be triggered and only the stores in a specific city will be showed.
The json response will be the same in the two handler for yes and no reply.
{
"payload": {
"google": {
"expectUserResponse": true,
"richResponse": {
"items": [
{
"simpleResponse": {
"textToSpeech": "Store near: Address Number City.\n Now: OPEN."
}
},
{
"carouselBrowse": {
"items": [
{
"title": "title 0",
"openUrlAction": {
"url": "https://website.it/?0"
},
"description": "description",
"image": {
"url": "https://avatars3.githubusercontent.com/u/5048136?s=460&v=4",
"accessibilityText": "empty"
}
},
{
"title": "title 1",
"openUrlAction": {
"url": "https://website.it/?1"
},
"description": "description",
"image": {
"url": "https://avatars3.githubusercontent.com/u/5048136?s=460&v=4",
"accessibilityText": "empty"
}
},
{
"title": "title 2",
"openUrlAction": {
"url": "https://website.it/?2"
},
"description": "description",
"image": {
"url": "https://avatars3.githubusercontent.com/u/5048136?s=460&v=4",
"accessibilityText": "empty"
}
}
]
}
}
]
},
"userStorage": "{\"lat\":45.4627124, \"long\": 9.1076928}"
}
},
"outputContexts": [
{
"name": "projects/project-name/agent/sessions/ABppEePAPYRhvT9Pcwmu3S61Ka12DUN5gmem7v0p/contexts/context-name",
"lifespanCount": 1,
"parameters": {
"Data": ""
}
}
],
"followupEventInput": {
"parameters": {
"data": {
"listSelect": {}
}
}
}
}
Problem
When I reply "no", the BrowseCarousel works. When I reply "yes", the BrowseCarousel does not work.
I cannot find the reason of this. The json response is exactly the same in the two different intents.

Issue solved.
The answer is: the simulator is broken for this particular event. When using a real device the browsecarousel is working as expected.
USE A REAL DEVICE IF YOU WANT TO SEE EXACTLY WHAT THE RESULT LOOK LIKE.

Related

Alexa CBT Test: Failed Test due to DeepQuery=True

My colleagues and I have been working to fix a reported issue on our Amazon Alexa CBT Test regarding the value “DeepQuery=true”.
Our code has been modified, so that every state change is reported automatically and all the used interfaces have the properties “proactivelyReported” and “retrievable” set to true.
As has been suggested by the WWA-Support we used the Smart Home Debugger of the Developer Console to validate the ReportEvents (e.g. Discovery or ChangeReport) and we checked the state of our device on the “View Device State” page (both pages are referenced on: https://developer.amazon.com/en-US/docs/alexa/smarthome/debug-your-smart-home-skill.html).
For debugging purposes we scaled our device capabilities down to just the PowerController. The AddOrUpdateReport of Alexa.Discovery looks to our eyes now exactly as expected/documented. Same goes for the ChangeReport, which we proactively send right after the AddOrUpdateReport (Two sample-Reports for both are provided at the end).
Unfortunately we are still faced with the issue, that “DeepQuery=true” on the “View Device State” page.
If we set the interface property “retrievable” to false, “DeepQuery=false”, but the Alexa-App does not retain the current state of the device. In this configuration the Alexa-App can only be used to send commands, which unfortunately will lead to other test cases to fail.
Does anyone know how to solve this issue?
How can we set “proactivelyReported” and “retrievable” to true and have “DeepQuery=false”?
Any help would be greatly appreciated and I will gladly provide more informations if needed.
Sample AddOrUpdateReport from Smart Home Debugger
{
"header": {
"namespace": "SkillDebugger",
"name": "CaptureDebuggingInfo",
"messageId": "05b030fb-6393-4ae0-80d0-47fc27876f0e"
},
"payload": {
"skillId": "amzn1.ask.skill.055ca62d-3cf8-4f51-a683-9a98b36f4637",
"timestamp": "2021-09-09T13:28:21.629Z",
"dialogRequestId": null,
"skillRequestId": null,
"type": "SmartHomeAddOrUpdateReportSuccess",
"content": {
"addOrUpdateReport": {
"event": {
"header": {
"namespace": "Alexa.Discovery",
"name": "AddOrUpdateReport",
"messageId": "2458b969-7c3e-47e2-ab0b-6e13a999be76",
"payloadVersion": "3"
},
"payload": {
"endpoints": [
{
"manufacturerName": "Our Company Name",
"description": "Our Product Name",
"endpointId": "device--cb12b420-1171-11ec-81f3-cb34e87ea438",
"friendlyName": "Lampe 1",
"capabilities": [
{
"type": "AlexaInterface",
"version": "3",
"interface": "Alexa.PowerController",
"properties": {
"supported": [
{
"name": "powerState"
}
],
"proactivelyReported": true,
"retrievable": true
}
},
{
"type": "AlexaInterface",
"interface": "Alexa",
"version": "3"
}
],
"displayCategories": [
"LIGHT"
],
"connections": [],
"relationships": {},
"cookie": {}
}
],
"scope": null
}
}
}
}
}
}
Sample ChangeReport from Smart Home Debugger
{
"header": {
"namespace": "SkillDebugger",
"name": "CaptureDebuggingInfo",
"messageId": "194a96a1-6747-46ba-8751-5c9ef715fd34"
},
"payload": {
"skillId": "amzn1.ask.skill.055ca62d-3cf8-4f51-a683-9a98b36f4637",
"timestamp": "2021-09-09T13:28:23.227Z",
"dialogRequestId": null,
"skillRequestId": null,
"type": "SmartHomeChangeReportSuccess",
"content": {
"changeReport": {
"event": {
"header": {
"namespace": "Alexa",
"name": "ChangeReport",
"messageId": "8972e386-9622-40e6-85e7-1a7d81c79c8a",
"payloadVersion": "3"
},
"endpoint": {
"scope": null,
"endpointId": "device--cb12b420-1171-11ec-81f3-cb34e87ea438"
},
"payload": {
"change": {
"cause": {
"type": "APP_INTERACTION"
},
"properties": [
{
"namespace": "Alexa.PowerController",
"name": "powerState",
"value": "ON",
"timeOfSample": "2021-09-09T13:28:18.088Z",
"uncertaintyInMilliseconds": 500
}
]
}
}
},
"context": {
"properties": []
}
}
}
}
}

How to make a carousel using Google.Cloud.Dialogflow.V2 Carousel for dialogflow?

Im trying to build a webhook for dialogflow with help from nuget package Google.Cloud.Dialogflow.V2.
Im running the code with .Net Core 2.1.
I get the fullfilmentText to show in dialogflow, but cant make the carousel to show up.
What am I doing wrong?
Here is my method:
[HttpPost]
public ContentResult Post([FromBody] WebhookRequest webhookRequest)
{
var dialogflowResponse = new WebhookResponse { FulfillmentText = "FulfillmentText" };
var carousel = new Intent.Types.Message.Types.CarouselSelect.Types.Item
{
Title = "My title",
Description = "This is a description",
Image = new Intent.Types.Message.Types.Image
{
ImageUri = "https://placekitten.com/200/300",
AccessibilityText = "cat"
},
Info = new Intent.Types.Message.Types.SelectItemInfo
{
Key = "Cats"
}
};
carousel.Info.Synonyms.Add("Synonym 1");
carousel.Info.Synonyms.Add("Synonym 1");
dialogflowResponse.FulfillmentMessages.Add(new Intent.Types.Message {
CarouselSelect = new Intent.Types.Message.Types.CarouselSelect()
});
dialogflowResponse.FulfillmentMessages.First().CarouselSelect.Items.Add(carousel);
return dialogflowResponse;
}
Generates followiing Json:
{
"fulfillmentText": "FulfillmentText",
"fulfillmentMessages": [
{
"carouselSelect": {
"items": [
{
"info": {
"key": "Cats",
"synonyms": [
"Synonym 1",
"Synonym 1"
]
},
"title": "My title",
"description": "This is a description",
"image": {
"imageUri": "https://placekitten.com/200/300",
"accessibilityText": "cat"
}
}
]
}
}
]
}
i ran into the same problem with the java api. Only the basic rich messages (card, quick replies, ...) are working. BasicCard, Suggestions, Carousel, ... doesn't show up in any channel.
I got it to work if i added some of the following custom payload to skype channel:
{
"skype": {
"type": "message",
"attachmentLayout": "carousel",
"text": "Please select your food.",
"attachments": [{
"contentType": "application/vnd.microsoft.card.hero",
"content": {
"title": "Sushi",
"subtitle": "Very fresh shrimp or tuna.",
"images": [{
"url": "https://i.ibb.co/87LmvT3/mdi.png"
}
],
"buttons": [{
"type": "imBack",
"title": "1 piece",
"value": "sushi,1"
}, {
"type": "imBack",
"title": "2 piece",
"value": "sushi,2"
}, {
"type": "imBack",
"title": "more",
"value": "sushi,3+"
}
]
}
}, {
"contentType": "application/vnd.microsoft.card.hero",
"content": {
"title": "Tenpura",
"subtitle": "Japanese first-class vegitables.",
"images": [{
"url": "https://i.ibb.co/87LmvT3/mdi.png"
}
],
"buttons": [{
"type": "imBack",
"title": "1 piece",
"value": "tenpura,1"
}, {
"type": "imBack",
"title": "2 piece",
"value": "tenpura,2"
}, {
"type": "imBack",
"title": "more",
"value": "tenpura,3+"
}
]
}
}
]
}}
I hope it helps!

carouselSelect in V2 - everything like in doc but it doesn't work

I read a documentation and saw the format of carousel Select (API V2)
"fulfillmentMessages": [
{
"platform": "FACEBOOK",
"carouselSelect": {
"items": [
{
"info": {
"key": "Pologne"
},
"title": "Varsovie",
"description": "Prices are starting from 204",
"image": {
"imageUri": "https://bucketeer-2aea38ff-6103-4b82-a7e2-760c5feac625.s3.amazonaws.com/public/url_z/PL-sky/WARS-sky/1.jpg"
}
},
{
"info": {
"key": "Turquie"
},
"title": "Istanbul",
"description": "Prices are starting from 229",
"image": {
"imageUri": "https://bucketeer-2aea38ff-6103-4b82-a7e2-760c5feac625.s3.amazonaws.com/public/url_z/TR-sky/ISTA-sky/1.jpg"
}
}
]
}
},
{
"platform": "SLACK",
"carouselSelect": {
"items": [
{
"info": {
"key": "Pologne"
},
"title": "Varsovie",
"description": "Prices are starting from 204",
"image": {
"imageUri": "https://bucketeer-2aea38ff-6103-4b82-a7e2-760c5feac625.s3.amazonaws.com/public/url_z/PL-sky/WARS-sky/1.jpg"
}
},
{
"info": {
"key": "Turquie"
},
"title": "Istanbul",
"description": "Prices are starting from 229",
"image": {
"imageUri": "https://bucketeer-2aea38ff-6103-4b82-a7e2-760c5feac625.s3.amazonaws.com/public/url_z/TR-sky/ISTA-sky/1.jpg"
}
}
]
}
}
],
But it, unfortunately doesn't work. Do you have any ideas why? When I use Card object from the doc is worked fine. Also if I add fulfillmentText the bot displays text
You can read more about what types of rich messages Dialogflow supports here in the docs. Dialogflow supports Text, Cards, Images, Suggestion Chips (Quick Replies), and Payloads (Platform-specific responses).
For the content of the custom payload please see the target platform documentation (Google Assistant, Slack, Facebook messenger, etc.)
Actions on Google, however, supports more rich responses, such as carousels, etc.

webhook doesn't receive event from m.me link

I've been trying to pass parameter using m.me link, but when tester/ developer account of my app tried this feature, my webhook didn't receive any event.
My bot already has a get started button and the page is published.
what im trying to ask is why couldn't i get any event and how to solve this problem?
let message = {
"attachment": {
"type": "template",
"payload": {
"template_type": "generic",
"elements": [
{
"title": "Share me 😊",
"subtitle": "share to your friend",
"image_url": "https://gallery.yopriceville.com/var/resizes/Free-Clipart-Pictures/Sun-and-Moon-PNG/Transparent_Sun_with_Shades_PNG_Clipart_Picture.png?m=1434276589",
"buttons": [
{
"type": "element_share",
"share_contents": {
"attachment": {
"type": "template",
"payload": {
"template_type": "generic",
"elements": [
{
"title": "Add to connect to your friend",
"subtitle": "connect",
"image_url": output.profile_pic,
"default_action": {
"type": "web_url",
"url": `http://m.me/${page_id}?ref=invited_by_${recipientId}`
},
"buttons": [
{
"type": "web_url",
"url": `http://m.me/${page_id}?ref=invited_by_${recipientId}`,
"title": `ADD ${output.first_name}`
}
]
}
]
}
}
}
}
]
}
]
}
}
};

List_card in AoG passing item title to the next query instead of key

In an app, I'm returning messages of type list_card with option key for an intent.
Here is the json of a sample query:
{
"id": "275212ef-cf97-4576-afa7-facfbc044ada",
"timestamp": "2017-07-17T17:36:03.655Z",
"lang": "en",
"result": {
"source": "agent",
"resolvedQuery": "who is Sneha",
"action": "cp.name_search",
"actionIncomplete": false,
"parameters": {
"keyword": "Sneha"
},
"contexts": [
{
"name": "cpname_search-followup",
"parameters": {
"keyword.original": "Sneha",
"keyword": "Sneha"
},
"lifespan": 2
},
{
"name": "cpuid_search-followup",
"parameters": {
"keyword.original": "Sneha",
"keyword": "Sneha"
},
"lifespan": 1
}
],
"metadata": {
"intentId": "86bd1a17-8e9a-4956-b270-5fb4ac952f5f",
"webhookUsed": "true",
"webhookForSlotFillingUsed": "false",
"webhookResponseTime": 135,
"intentName": "cp.name_search"
},
"fulfillment": {
"speech": "Searching...",
"source": "agent",
"messages": [
{
"type": "simple_response",
"platform": "google",
"textToSpeech": "Here are the search results. \nWant anything else?"
},
{
"type": "list_card",
"platform": "google",
"title": "Search results",
"items": [
{
"optionInfo": {
"key": "uid 72",
"synonyms": []
},
"title": "Sneha Vasista",
"description": "Srinivas Institute of Technology",
"image": {
"url": "//www.curlpad.com/assets/img/custom_images/user.png"
}
},
{
"optionInfo": {
"key": "uid 2053",
"synonyms": []
},
"title": "Sneha Bhat",
"description": "Canara Engineering College",
"image": {
"url": "//www.curlpad.com/assets/img/custom_images/user.png"
}
},
{
"optionInfo": {
"key": "uid 2114",
"synonyms": []
},
"title": "Sneha Sajan",
"description": "P.A College of Engineering",
"image": {
"url": "//www.curlpad.com/assets/img/custom_images/user.png"
}
},
{
"optionInfo": {
"key": "uid 2320",
"synonyms": []
},
"title": "Sneha ",
"description": "sdit",
"image": {
"url": "//www.curlpad.com/assets/img/custom_images/user.png"
}
},
{
"optionInfo": {
"key": "uid 2363",
"synonyms": []
},
"title": "Sneha ",
"description": "Srinivas School of Engineering, Mukka",
"image": {
"url": "//www.curlpad.com/assets/img/custom_images/user.png"
}
}
]
},
{
"type": "0",
"speech": "Here are the search results."
}
]
},
"score": 1
},
"status": {
"code": 200,
"errorType": "success"
},
"sessionId": "e6aa9e52-a9e1-481a-adb5-476c5b386e02"
}
Now the problem is, when I tap the list item in AoG simulator, it passes title of item to next query.
But while testing in Api.ai simulator, it behaves well and passes that key to the next query.
What can be the problem here?
Any hints?
If you're using API.AI, then it will appear in the Intent as an actions_intent_OPTION Event.
One good solution is to have the Intent that sends the list with an OutputContext. Then create a particular Fallback Intent with actions_intent_OPTION as Event and your desired action for that Context which should handle both the voice and press responses.
And you will find your option_key at ["originalRequest"]["data"]["inputs"][0]["arguments"][0]["textValue"] instead of parameter.
You can also see the response value in the actions_intent_option context.
What you need to do is set up a fallback intent directly under your current intent.
For example, if you display the list from the default welcome intent, you can do the following.
Click "Add follow-up intent" and choose fallback.
Don't forget to set the action and enable webhook in the fallback intent.
Now, you should be able to retrieve your answer from the fallback intent using the following code.
const param = app.getContextArgument('actions_intent_option','OPTION').value;

Resources