DialogFlow Webhook works in DF but not Google Assistant Simulator - node.js

I've resorted to returning a default response in order to try to get this to work. This is hosted on AWS Lambda with an API Gateway. Last night I had some different code working, but now I can't seem to get anything to work through Google Assistant.
Here's the example V2 response I am using:
callback(null, {
"payload": {
"google": {
"expectUserResponse": true,
"richResponse": {
"items": [
{
"simpleResponse": {
"textToSpeech": "Choose a item"
}
}
]
},
"systemIntent": {
"intent": "actions.intent.OPTION",
"data": {
"#type": "type.googleapis.com/google.actions.v2.OptionValueSpec",
"listSelect": {
"title": "Hello",
"items": [
{
"optionInfo": {
"key": "first title key"
},
"description": "first description",
"image": {
"url": "https://developers.google.com/actions/images/badges/XPM_BADGING_GoogleAssistant_VER.png",
"accessibilityText": "first alt"
},
"title": "first title"
},
{
"optionInfo": {
"key": "second"
},
"description": "second description",
"image": {
"url": "https://lh3.googleusercontent.com/Nu3a6F80WfixUqf_ec_vgXy_c0-0r4VLJRXjVFF_X_CIilEu8B9fT35qyTEj_PEsKw",
"accessibilityText": "second alt"
},
"title": "second title"
}
]
}
}
}
}
}
});
Here's the log output from Google Assistant to DialogFlow:
Received response from agent with body: HTTP/1.1 200 OK Server: nginx/1.13.6 Date: Sat, 11 May 2019 18:44:24 GMT Content-Type: application/json;charset=UTF-8 Content-Length: 772 X-Cloud-Trace-Context: e62f0526a5238882dd1c1a3b3a70e3b5/11459115340038791606;o=0 Google-Actions-API-Version: 2 Assistant-Interaction-Error-Code: -1 Assistant-Interaction-Error-Message: Failed to parse Dialogflow response into AppResponse because of empty speech response X-SHARD: default Via: 1.1 google Alt-Svc: clear
{
"responseMetadata": {
"status": {
"code": 10,
"message": "Failed to parse Dialogflow response into AppResponse because of empty speech response",
"details": [
{
"#type": "type.googleapis.com/google.protobuf.Value",
"value": "{\"id\":\"99120b53-f0a7-415f-b44e-73c7525fcd43\",\"timestamp\":\"2019-05-11T18:44:24.422Z\",\"lang\":\"en-us\",\"result\":{},\"alternateResult\":{},\"status\":{\"code\":206,\"errorType\":\"partial_content\",\"errorDetails\":\"Webhook call failed. Error: Failed to parse webhook JSON response: Cannot find field: errorMessage in message google.cloud.dialogflow.v2.WebhookResponse.\"},\"sessionId\":\"ABwppHHok5GTHl8A6XWq9q-Yp1chkoZ4vT688i4HTxVgVQDcY1zHH2JUqVeCMPc_6bbA7WFf1e-nw0ZQxQ\"}"
}
]
}
}
}
I'm not sure where to go from here...Everything seems to be working correctly, I just can't figure out where the parser error is with Google Assistant. What gives?
Thanks.

Related

How to send to Amazon's Alexa Event Gateway?

I am trying to test sending an event to the Amazon's Event Gateway for my Alexa Smart Home skill using Postman but I keep receiving an 'invalid access token exception.' I have read the Amazon's documentation on this but apparently I am missing something.
When I enable my skill, my Smart Home Lambda receives the AcceptGrant.
{
"directive": {
"header": {
"namespace": "Alexa.Authorization",
"name": "AcceptGrant",
"messageId": "b2862179-bc56-4bb2-ac05-ce55c7a3e977",
"payloadVersion": "3"
},
"payload": {
"grant": {
"type": "OAuth2.AuthorizationCode",
"code": "ANSVjPzpTDBsdfoRSyrs"
},
"grantee": {
"type": "BearerToken",
"token": "Atza|IwEB..."
}
}
}
}
My lambda sends a POST to 'https://api.amazon.com/auth/o2/token' to receive the Access and Refresh tokens. It then stores those tokens. Next, my Lamdba responds with the following:
{
"event": {
"header": {
"namespace": "Alexa.Authorization",
"name": "AcceptGrant.Response",
"messageId": "b2862179-bc56-4bb2-ac05-ce55c7a3e977",
"payloadVersion": "3"
},
"payload": {}
}
}
I then get a message web page that I have successfully linked my skill - all is good.
Next, I try to send an event to Amazon's Alexa event gateway using the Postman app. I put the Access token (I also tried the Refresh token) in the header as a 'BearerToken' type and the in the 'scope' of the 'endpoint' object.
POST https://api.amazonalexa.com/v3/events?Content-Type=application/json&charset=UTF-8
with a header that specifies a Bearer Token (Access token received earlier) and a body that contains the following:
{
"event": {
"header": {
"messageId": "abc-123-def-456",
"namespace": "Alexa",
"name": "ChangeReport",
"payloadVersion": "3"
},
"endpoint": {
"scope": {
"type": "BearerToken",
"token": "<access token>"
},
"endpointId": "MySmartSwitch-001"
},
"payload": {
"change": {
"cause": {
"type": "RULE_TRIGGER"
},
"properties": [
{
"namespace": "Alexa.ModeController",
"name": "mode",
"value": "Backup",
"timeOfSample": "2020-01-02T09:30:00ZZ",
"uncertaintyInMilliseconds": 50
}
]
}
}
},
"context": {
"properties": [
{
"namespace": "Alexa.PowerController",
"name": "powerState",
"value": "ON",
"timeOfSample": "2020-01-02T09:30:00Z",
"uncertaintyInMilliseconds": 60000
},
{
"namespace": "Alexa.EndpointHealth",
"name": "connectivity",
"value": {
"value": "OK"
},
"timeOfSample": "2020-01-02T09:30:00Z",
"uncertaintyInMilliseconds": 0
}
]
}
}
The response received is '401 Unauthorized'
{
"header": {
"namespace": "System",
"name": "Exception",
"messageId": "95bd23c3-76e6-472b-9c6d-74d436e1eb61"
},
"payload": {
"code": "INVALID_ACCESS_TOKEN_EXCEPTION",
"description": "Access token is not valid."
}
}
I figured out the issue. I was mistakenly sending parameters: Content-Type=application/json and charset=UTF-8 as well including them in the header - my bad. You just need to include them in the header.

Malformed Response: Failed to parse Dialogflow response into AppResponse because of empty speech response

I am using firebase function for the webhook fulfillment in Dialogflow. I am getting webhook successful as a fulfillment status but it is not working. I am using version 1. When I test it on Google Assistant simulator, it says "App is not responding".
firebase function
const functions = require('firebase-functions');
exports.webhook = functions.https.onRequest((request, response) => {
response.send({
"google":{
"richResponse":{
"items":[
{
"simpleResponse":{
"textToSpeech":"Hey! Good to see you."
}
},
{
"mediaResponse":{
"mediaType":"AUDIO",
"mediaObjects":[
{
"name":"Exercises",
"description":"ex",
"largeImage":{
"url":"http://res.freestockphotos.biz/pictures/17/17903-balloons-pv.jpg",
"accessibilityText":"..."
},
"contentUrl":"https://theislam360.me:8080/hbd.mp3"
}
]
}
}
],
"suggestions":[
{
"title":"chips"
}
]
}
}
}
)
});`
When I copy paste the response from {google... to the end in the custom payload manually via GUI, It works. While for webhook, it is not working.
RAW API RESPONSE
{
"id": "eaf627ed-26b5-4965-b0b0-bc77144e144b",
"timestamp": "2019-04-15T11:54:18.948Z",
"lang": "en",
"result": {
"source": "agent",
"resolvedQuery": "play hbd",
"action": "",
"actionIncomplete": false,
"parameters": {
"any": "hbd"
},
"contexts": [],
"metadata": {
"isFallbackIntent": "false",
"webhookResponseTime": 34,
"intentName": "play",
"intentId": "e60071cd-ce31-4ef9-ae9b-cc370c3362b3",
"webhookUsed": "true",
"webhookForSlotFillingUsed": "false"
},
"fulfillment": {
"messages": []
},
"score": 1
},
"status": {
"code": 200,
"errorType": "success"
},
"sessionId": "e91bd62f-766b-b19d-d37b-2917ac20caa6"
}
FULFILLMENT REQUEST
{
"id": "eaf627ed-26b5-4965-b0b0-bc77144e144b",
"timestamp": "2019-04-15T11:54:18.948Z",
"lang": "en",
"result": {
"source": "agent",
"resolvedQuery": "play hbd",
"speech": "",
"action": "",
"actionIncomplete": false,
"parameters": {
"any": "hbd"
},
"contexts": [],
"metadata": {
"intentId": "e60071cd-ce31-4ef9-ae9b-cc370c3362b3",
"webhookUsed": "true",
"webhookForSlotFillingUsed": "false",
"isFallbackIntent": "false",
"intentName": "play"
},
"fulfillment": {
"speech": "",
"messages": []
},
"score": 1
},
"status": {
"code": 200,
"errorType": "success"
},
"sessionId": "e91bd62f-766b-b19d-d37b-2917ac20caa6"
}
FULFILLMENT RESPONSE
{
"google": {
"richResponse": {
"items": [
{
"simpleResponse": {
"textToSpeech": "Hey! Good to see you."
}
},
{
"mediaResponse": {
"mediaType": "AUDIO",
"mediaObjects": [
{
"name": "Exercises",
"description": "ex",
"largeImage": {
"url": "http://res.freestockphotos.biz/pictures/17/17903-balloons-pv.jpg",
"accessibilityText": "..."
},
"contentUrl": "https://theislam360.me:8080/hbd.mp3"
}
]
}
}
],
"suggestions": [
{
"title": "chips"
}
]
}
}
}
FULFILLMENT STATUS
Webhook execution successful
Firebase Logs
Google Assistant Simulator Logs
You're not using the correct JSON in the response. By putting it in the GUI in the "custom payload" section, it is creating a larger JSON response for you. The google object needs to be under the data object for Dialogflow v1 or payload for Dialogflow v2. (And if you haven't switched to v2 - you should do so immediately, since v1 will be switched off in about a month.)
So what you're returning should look more like
{
"payload": {
"google": {
...
}
}
}

There was a problem with the requested skill's response using template, Alexa skill using template directive

I have an issue with alexa skill in node.
i'm making an skill that connect with db via HTTS.response
everything works fine until I want to show the results in a template.
this its the response.
{
"outputSpeech": {
"type": "PlainText",
"text": "That's sound good. Your search is Property Type: residential - City:Petersburg - Zip Code:33715 - Bedrooms:2 - Price: $100000"
},
"directives": [
{
"type": "Display.RenderTemplate",
"template": {
"type": "ListTemplate2",
"token": "ShowHomeView",
"backButton": "hidden",
"backgroundImage": {
"contentDescription": "",
"sources": [
{
"url": "https://s3.amazonaws_com/ask-samples-resources/berryImages/main_blur2.png"
}
]
},
"title": "Search Results",
"listItems": [
{
"token": 1,
"image": "https://mydomain/images/fine_arts-1550.jpg",
"textContent": "This is my list item 1"
},
{
"token": 2,
"image": "https://mydomain/images/fine_arts-1550.jpg",
"textContent": "This is my list item 2"
},
{
"token": 3,
"image": "https://mydomain/images/fine_arts-1550.jpg",
"textContent": "This is my list item 3"
}
]
}
},
{
"type": "Hint",
"hint": {
"type": "PlainText",
"text": " select number two. "
}
}
],
"shouldEndSession": false
}
and in alexa console say:
There was a problem with the requested skill's response
and in the device log say.
"descriptiveText": [
"Request Identifier: amzn1.echo-api.request.f6d03bab-05b2-449f-877b-b58f8e28274b",
"",
"The skill returned an invalid response"
],
What could be causing this error?
thnks in advance

Dialogflow v2 API - cards not shown in the simulator

I've a webhook for fulfillment.
Below is the code that's responding back
let result_obj = {
"fulfillmentText": "This is a text response",
"fulfillmentMessages": [
{
"text": {
"text": [
"this is test"
]
}
},
{
"card": {
"title": "card title",
"subtitle": "card text",
"imageUri": "https://assistant.google.com/static/images/molecule/Molecule-Formation-stop.png",
"buttons": [
{
"text": "button text",
"postback": "https://assistant.google.com/"
}
]
}
}
]
}
Below is the result from dialogflow GUI
Below is what I get when I run from the simulator or from the Google Assistant application on the Android phone
Both the simulator and phone are not showing the cards. Am I missing something obvious here?
For rich responses like cards to show on Google Assistant you have to use the payload part of response JSON, here is an example:
{
"fulfillmentText": "This is a text response",
"fulfillmentMessages": [],
"source": "example.com",
"payload": {
"google": {
"expectUserResponse": true,
"richResponse": {
"items": [
{
"simpleResponse": {
"textToSpeech": "This is a Basic Card:"
}
},
{
"basicCard": {
"title": "card title",
"image": {
"url": "https://www.google.com/images/branding/googlelogo/2x/googlelogo_color_272x92dp.png",
"accessibilityText": "Google Logo"
},
"buttons": [
{
"title": "Button Title",
"openUrlAction": {
"url": "https://www.google.com"
}
}
],
"imageDisplayOptions": "WHITE"
}
}
]
}
}
},
"outputContexts": [],
"followupEventInput": {}
}
Check out this github repo for all rich-responses' JSON formats.

Webhook call failed. Error: Failed to parse webhook JSON response: Cannot find field: messages in message google.cloud.dialogflow.v2.Intent.Message

I built a bot using Dialogflow and connected it to a local webhook (now accessing it through ngrok). I am able to receive the response from Dialogflow but I am unable to replay to it. I followed the JSON structure as shown here - Test response from webhook. But I am getting the following error in Dialogflow.
Webhook call failed. Error: Failed to parse webhook JSON response:
Cannot find field: messages in message
google.cloud.dialogflow.v2.Intent.Message.
Following is the reply that I sent to Dialogflow -
{
"messages":[
{
"speech":"Text response",
"type":0
}
]
}
Please tell me what should be the exact format of the reply that I should send to Dialogflow.
From v1 to v2, the response object almost change completely. For just simple text, you can use like:
{
"fulfillmentText": "Text response",
"fulfillmentMessages": [
{
"text": {
"text": ["Text response"]
}
}
],
"source": "<Text response>"
}
I faced same issue,resolved using below json on dialogflow :
I made a simple node program which accepts a post response and returns json of the format accepted by Dialogflow.You may send your request in any way you like. check on Fulfillment status tab :
The field messageswas renamed/refactored to fulfillmentMessages - "can not find" means that it is not a property in the definition.
This is some comparable result accepted by v2:
{
"fulfillmentText": "response text",
"fulfillmentMessages": [{"simpleResponses": {"simpleResponses": [ {
"textToSpeech": "response text",
"displayText": "response text"
}]}}]
}
Messages alone is not sufficient. Refer to Dialogflow V2 webhook fulfillment documentation for complete list of parameters expected and format of JSON.
Are you sure you are using V2 of the DialogFlow APIs?
Use the Webhook Playground to Get the appropriate response for either the Dialogflow API or the Actions SDK which is depricated but still works. There is also the yet newer and different API for the Google Actions Builder/SDK framework as per :
DiaglowFlow JSON Response:
{
"payload": {
"google": {
"expectUserResponse": true,
"richResponse": {
"items": [
{
"simpleResponse": {
"textToSpeech": "Webhook worked.",
"displayText": "Webhook worked."
}
}
]
}
}
}
}
Actions SDK Response:
{
"expectUserResponse": true,
"expectedInputs": [
{
"inputPrompt": {
"richInitialPrompt": {
"items": [
{
"simpleResponse": {
"textToSpeech": "Webhook worked.",
"displayText": "Webhook worked."
}
}
]
}
},
"possibleIntents": [
{
"intent": "actions.intent.TEXT"
}
]
}
]
}
Action Builder/SDK. Note that the session id needs to be returned.
{
"session": {
"id": "example_session_id",
"params": {}
},
"prompt": {
"override": false,
"firstSimple": {
"speech": "Hello World.",
"text": ""
}
},
"scene": {
"name": "SceneName",
"slots": {},
"next": {
"name": "actions.scene.END_CONVERSATION"
}
}
}

Resources