I'm building a skill with the node.js SDK for the Echo Show. I want to use the VideoApp feature (documentation) to display a video based on information I elicit from the user. I have the video in S3, and constructed the directive and the entire response object, and called "response ready" like so:
var directive = [
{
"type": "VideoApp.Launch",
"videoItem": {
"source": "https://s3.amazonaws.com/path/to/video.mp4",
"metadata": {
"title": "Video Title",
"subtitle": "Subtitle to video"
}
}
}
];
this.handler.response = buildResponse(directive);
this.emit(':responseReady');
I expected the Echo Show to use the response object I generated to display my video, but instead it says "there was a problem with the requested skill's response." It also displays "Invalid directive" in the corner when it says that. Below is the full response object I generate, any help on how to properly launch a video would be appreciated!
{
"version": "1.0",
"response": {
"shouldEndSession": true,
"outputSpeech": null,
"reprompt": null,
"directives": [
{
"type": "VideoApp.Launch",
"videoItem": {
"source": "https://s3.amazonaws.com/path/to/video.mp4",
"metadata": {
"title": "Video title",
"subtitle": "Subtitle to video"
}
}
}
],
"card": null
}
}
Also I've found that you get invalid responses if the shouldEndSession attribute is included with the VideoApp.Launch directive.
I figured it out. I just had to make the video, and bucket it was residing in, publicly readable in S3 permissions.
Related
I am 500% sure it used to work and all of a sudden this is broken. The card for getting input is no longer passing the value back to nodejs.
The card looks like below:
{
"type": "AdaptiveCard",
"body": [
{
"type": "TextBlock",
"text": "Note text"
},
{
"type": "Input.Text",
"placeholder": "Type a note",
"isMultiline": true,
"id": "noteIdVal"
}
],
"actions": [
{
"type": "Action.Submit",
"title": "Save",
"data": { "action" : "add_note", "objNumber": objId, "objType": objectType }
},
{
"type": "Action.Submit",
"title": "Cancel",
"data" : {"action": "cancel"}
}
],
"$schema": "http://adaptivecards.io/schemas/adaptive-card.json"
}
On the submit action, in my nodejs i am getting the data in the values node which are passed in the data field. However, it is no longer attaching noteIdVal. Did something changed from MS side?
MS Teams Adaptive card required special property with the name msteams to the object in an object submit action’s data property in order to access this functionality.
{
"type": "Action.Submit",
"title": "Click me for messageBack",
"data": {
"msteams": {
"type": "messageBack",
"displayText": "I clicked this button",
"text": "text to bots",
"value": "{\"bfKey\": \"bfVal\", \"conflictKey\": \"from value\"}"
},
"extraData": {}
}
}
The type property is "messageBack" the submit action will behave like a messageBack card action, which is like a combination of imBack and postBack.
Reference :
Microsoft Docs for MS Teams Adaptive Card
So, may be useful to other folks here. I have two showCards and the content of both the show cards has a common text field with same id name "noteIdVal". As ultimately it is a single json and hence was the culprit.
Lesson, have all fields unique id values which is easy to miss when you have multiple show cards
I'm working on a very simple Dialogflow with about 15-20 intents. All of these intents use a text response except one. The only intent that does not use a text response is called 'repeat'. The intent (repeat) should be able to repeat whatever was previously said by Google Assistant.
I've tried to set this up using Multivocal but have not been successful. When I type a command into the test simulator I'll get the initial response, but when I follow up with 'repeat' the default response of 'Not available' is returned. The webhook times out when I look at the Diagnostic Info. My sense is that I've configured something wrong because I've read these answers and not been able to solve my problem:
How to repeat last response of bot in dialogflow
Dialogflow - Repeat last sentence (voice) for Social Robot Elderly
Use multivocal libary to configure repeat intent in Dialogflow for VUI
I'm using the inline editor within Dialogflow my index.js looks like:
const Multivocal = require('multivocal');
const conf = {
Local: {
en: {
Response: {
"Action.multivocal.repeat": "Let me try again",
}
}
}
};
new Multivocal.Config.Simple( conf );
exports.webhook = Multivocal.processFirebaseWebhook;
exports.dialogflowFirebaseFulfillment = Multivocal.processFirebaseWebhook;
And my package.json includes the Multivocal dependency:
"multivocal": "^0.15.0"
My understanding based on the above SO questions is that these config values would be enough and I don't need to do any coding, but I'm clearly screwing something (many things?) up. How can I get the prior response in Google Assistant to repeat when a user says 'repeat' or something similar? Multivocal seems like a simple solution, if I can do it that way.
Additional logs:
Fulfillment request (removed project id information):
{
"responseId": "--",
"queryResult": {
"queryText": "repeat",
"action": "multivocal.repeat",
"parameters": {},
"allRequiredParamsPresent": true,
"fulfillmentMessages": [
{
"text": {
"text": [
""
]
}
}
],
"outputContexts": [
{
"name": "project info",
"parameters": {
"no-input": 0,
"no-match": 0
}
}
],
"intent": {
"name": "project info",
"displayName": "repeat"
},
"intentDetectionConfidence": 1,
"languageCode": "en"
},
"originalDetectIntentRequest": {
"payload": {}
},
"session": "project info"
}
Raw API response (removed project and response id)
{
"responseId": "",
"queryResult": {
"queryText": "repeat",
"action": "multivocal.repeat",
"parameters": {},
"allRequiredParamsPresent": true,
"fulfillmentMessages": [
{
"text": {
"text": [
""
]
}
}
],
"intent": {
"name": "projects info",
"displayName": "repeat"
},
"intentDetectionConfidence": 1,
"diagnosticInfo": {
"webhook_latency_ms": 4992
},
"languageCode": "en"
},
"webhookStatus": {
"code": 4,
"message": "Webhook call failed. Error: DEADLINE_EXCEEDED."
}
}
My simple intent that I've added based on the recommendation that for repeat to work on an intent it must use fulfillment and not based text response in Dialogflow
Here is my index.js file using the inline editor with suggestion to add text response in the config:
const conf = {
Local: {
en: {
Response: {
"Intent.help": [
"I'm sorry, I'm not able to help you.",
"You, John, Paul, George, and Ringo ey?"
],
"Action.multivocal.repeat": "Let me try again"
}
}
}
};
This line at the end of my index.js seems odd to me, but may be unrelated:
exports.webhook = Multivocal.processFirebaseWebhook;
exports.dialogflowFirebaseFulfillment = Multivocal.processFirebaseWebhook;
It sounds like you're triggering the Fallback Intent. You also need an Intent defined in Dialogflow that has an Action set to "multivocal.repeat". That might look something like this:
In the dialogflow directory of the npm package (or on github) you'll find a zip file with this and several other "standard" Intents that you can use with mulivocal.
Additionally, all the other Intents that you want to be repeated must use fulfillment to send the response (the library doesn't know what might be sent unless it can send it itself). The simplest way to do this is to enable fulfillment on each, and move the text responses from their Dialogflow screens into the configuration under an entry such as "Intent.name" (replacing "name" with the name of the Intent) or "Action.name" if you set an action name for them.
So your configuration might be something like
const conf = {
Local: {
en: {
Response: {
"Intent.welcome": [
"Hello there!",
"Welcome to my Action!"
],
"Action.multivocal.repeat": [
"Let me try again"
]
}
}
}
};
So I have been trying to start off with creating chatbots on dialogflow. The issue I am running into is that my chatbot is meant to give users a bunch of options to select from and to take the conversation further from there. In order to implement that, I've used a suggestion chip I've included the JSON file that I am using. However, when testing, the bot detects the right intent but returns an empty response. I've included the code in case that helps.
{
"richContent": [
[
{
"options": [
{
"text": "Chip 1",
"image": {
"src": {
"rawUrl": "https://example.com/images/logo.png"
}
},
"link": "https://example.com"
},
{
"link": "https://example.com",
"text": "Chip 2",
"image": {
"src": {
"rawUrl": "https://example.com/images/logo.png"
}
}
}
],
"type": "chips"
}
]
]
}
If you are testing it in the console, you definitely won't get any response, since there is some issue with the way console works. Use dialogflow messenger to test it. Anyways in the actual production you won't be using simulator to test it. Dialogflow console is a simulator which has some capability restrictions but it will work when you use the messenger.
I am trying to sent custom payload in an dialogflot intent. When i am selecting the custom payload option available under google assistant it gives the following predefined json format : -
{
"google": {
}
}
now i am not aware about what i need to put in there in order to get a response from here. Any guide will be helpful
There are some compulsory Keys to be added in Rich Response JSON.
You must have Suggestion Chips and a Simple Response to maintain the follow-up of your Action. AoG rejects any action with missing Suggestion Chips or Follow-Up Response.
Refer to this JSON for Basic Card Response:
{
"payload": {
"google": {
"expectUserResponse": true,
"richResponse": {
"items": [
{
"simpleResponse": {
"textToSpeech": "Here's an example of a basic card."
}
},
{
"basicCard": {
"title": "Title: this is a title",
"subtitle": "This is a subtitle",
"formattedText": "This is a basic card. Text in a basic card can include \"quotes\" and\n most other unicode characters including emojis. Basic cards also support\n some markdown formatting like *emphasis* or _italics_, **strong** or\n __bold__, and ***bold itallic*** or ___strong emphasis___ as well as other\n things like line \nbreaks",
"image": {
"url": "https://storage.googleapis.com/actionsresources/logo_assistant_2x_64dp.png",
"accessibilityText": "Image alternate text"
},
"buttons": [
{
"title": "This is a button",
"openUrlAction": {
"url": "https://assistant.google.com/"
}
}
],
"imageDisplayOptions": "CROPPED"
}
},
{
"simpleResponse": {
"textToSpeech": "Which response would you like to see next?"
}
}
]
}
}
}
}
You can refer to the specific Rich Response JSON for your Action in the following Documentation:
https://developers.google.com/assistant/conversational/rich-responses#df-json-basic-card
I have a response which has Simple Response with a BasicCard.The response from dialogflow to Google assistant is like this:
"payload": {
"google": {
"expectUserResponse": true,
"richResponse": {
"items": [
{
"simpleResponse": {
"textToSpeech": "Can you tell your TV's model number? (Eg. 42PFL7008S/12)",
"displayText": "Can you tell your TV's model number? (Eg. 42PFL7008S/12)"
},
"basicCard": {
"image": {
"url": "https://",
"accessibilityText": "The model number can be found here "
},
"title": "The model number can be found on here,
"subtitle": "It is also called Set-Type,"
}
}
],
"suggestions": []
}
}
}
In the response GA, Simulator I am getting the following error:
expected_inputs[0].input_prompt.rich_initial_prompt: the first element must be a 'simple_response', a 'structured_response' or a 'custom_response'.
GA response i dont see the simple response in the simulator..Somewhere the simple response is discarded..Is there any flaw in this response from DialogFlow?
The issue is that each object in the items[] array is supposed to have a single field which indicates what the type of the item is. So one item will have a simpleResponse attribute with corresponding values. Another item will have a basicCard attribute, with all the values for it.
You seem to have a single object in the items[] array with two attributes, simpleResponse and basicCard. Split them up into two objects in the array. Something like this:
"items": [
{
"simpleResponse": {
"textToSpeech": "Can you tell your TV's model number? (Eg. 42PFL7008S/12)",
"displayText": "Can you tell your TV's model number? (Eg. 42PFL7008S/12)"
}
},
{
"basicCard": {
"image": {
"url": "https://",
"accessibilityText": "The model number can be found here "
},
"title": "The model number can be found on here,
"subtitle": "It is also called Set-Type,"
}
}
],