Actions on Google - handling carousel responses from dialogflow - dialogflow-es

I've created a simple Google Assistant interface using DialogFlow with several Carousels that I want to be able to chain together. Whenever I touch a carousel option though, it always goes to the first Intent that has the actions_intent_OPTION event specified. I can get to all of my screens using voice commands, but I'm not sure how to process the touch commands to send the user to right Intent.
Current code in webhook:
const party = 'party';
const cocktail = 'cocktail';
const SELECTED_ITEM_RESPONSES = {
[party]: 'You selected party',
[cocktail]: 'You selected cocktail',
};
function carousel(agent) {
//agent.add(`Item selected`);
app.intent('actions.intent.OPTION', (conv, params, option) => {
let response = 'You did not select any item from the list or carousel';
if (option && SELECTED_ITEM_RESPONSES.hasOwnProperty(option)) {
response = SELECTED_ITEM_RESPONSES[option];
} else {
response = 'You selected an unknown item from the list or carousel';
}
conv.ask(response);
});
}
If I leave the agent.add() line in, then I get "Item selected"... but if I try to use the app.intent code, it says I'm just getting an empty speech response.
I was trying to create 1 intent called CarouselHandler to process all the menu selections. I used the sample code to call the carousel() function when that intent gets hit by the event.
let intentMap = new Map();
intentMap.set('Default Welcome Intent', welcome);
intentMap.set('Default Fallback Intent', fallback);
intentMap.set('CarouselHandler', carousel);
agent.handleRequest(intentMap);

You have several questions in here about using options. Let's try to clear a few things up.
Can I get a different Intent triggered for each option?
No. The way options are reported to Dialogflow is that all options will trigger the same Intent. You're responsible for looking at the option string sent and calling another function if you wish.
As you've noted, you need to create an Intent with the Event actions_intent_OPTION.
Your code to handle this might look something like this, although there are other ways to handle it:
app.intent('list.reply.click', (conv, params, option) => {
// Get the user's selection
// Compare the user's selections to each of the item's keys
if (!option) {
conv.ask('You did not select any item from the list or carousel');
} else if (option === 'OPTION_1') {
handleOption1( conv );
} else if (option === 'OPTION_2') {
handleOption2Or3( conv );
} else if (option === 'OPTION_3') {
handleOption2Or3( conv );
} else {
conv.ask('You selected an unknown item from the list, or carousel');
}
});
Can I get a different Intent triggered for each carousel?
Yes. To do this, when you send the carousel you will set an OutgoingContext and delete any other OutgoingContexts you created for a carousel (set their lifespan to 0). Then you will create an Intent that has this Context as an IncomingContext.
The code to send a carousel might look something like this if you're using the actions-on-google library
conv.ask("Here is menu 2");
conv.ask(new List({
title: "Menu 2",
items: {
"OPTION_1": {
title: "Option 1",
description: "Description 1"
},
"OPTION_2": {
title: "Option 2",
description: "Description 2"
},
"OPTION_3": {
title: "Option 3",
description: "Description 3"
},
}
});
conv.contexts.set("menu_2",99);
conv.contexts.delete("menu_1");
conv.contexts.delete("menu_3");
// Don't forget to add suggestions, too
If you're using the dialogflow-fulfillment library, it would be similar, although there are a few differences:
let conv = agent.conv();
conv.ask("Here is menu 2");
conv.ask(new List({
title: "Menu 2",
items: {
"OPTION_1": {
title: "Option 1",
description: "Description 1"
},
"OPTION_2": {
title: "Option 2",
description: "Description 2"
},
"OPTION_3": {
title: "Option 3",
description: "Description 3"
},
}
});
agent.add(conv);
agent.setContext({name:"menu_1", lifespan:0});
agent.setContext({name:"menu_2", lifespan:99});
agent.setContext({name:"menu_3", lifespan:0});
If you were using multivocal, the response configuration might look something like this:
{
Template: {
Text: "Here is menu 2",
Option: {
Type: "carousel",
Title: "Menu 2",
Items: [
{
Title: "Option 1",
Body: "Description 1"
},
{
Title: "Option 2",
Body: "Description 2"
},
{
Title: "Option 3",
Body: "Description 3"
}
]
}
},
Context: [
{
name: "menu_1",
lifetime: 0
},
{
name: "menu_2",
lifetime: 99
},
{
name: "menu_3",
lifetime: 0
}
]
}
The Intent that would capture this option suggestion might look something like this:
Your code to handle this would be similar as above, except using the different Intent name.
If there are overlapping options between the handlers, they could call the same function that actually does the work (again, as illustrated above).
How can I handle voice and option responses the same way?
AoG, in some cases, will use the voice response to trigger the option. This is what the aliases are for. But even beyond this, if you have Intents that catch phrases from the user and an Intent that works with the Options, all you need to do is have the fulfillment code call the same function.
Why doesn't the code work?
The line
app.intent('actions.intent.OPTION', (conv, params, option) => {
Probably doesn't do what you think it does. Unless this is the name for the Intent in Dialogflow, the string actions.intent.OPTION won't be seen in your handler. It is also how you register an Intent handler with the actions-on-google library.
It also looks like you're mixing the dialogflow-fulfillment library way of registering Intent handlers with the actions-on-google library way of registering Intent handlers through your carousel() function. Don't do this. (This may also be part of the cause about why replies aren't getting back correctly.)

Related

postback button in dialogflow messenger cx

Hello I'm trying to create a flow in dialogflow cx, where in case of multiple options I want my user to select 1 option where all the options are buttons.
I have used the default payload but not sure how can I send back which button got clicked to my webhook and return respective info, currently if I click on button it simply open example.com, if I exclude the link it opens same page in new tab.
{
"type": "button",
"icon": {
"type": "chevron_right",
"color": "#FF9800"
},
"text": "Button text 1",
"link" : "www.example.com",
"event": {
"name": "some name",
"languageCode": "en",
"parameters": {}
}
}
For your use case, since the button response type always redirects to a page when clicked, you can consider using suggestion chips instead.
{
"richContent": [
[
{
"options": [
{
"text": "Chip 1"
},
{
"text": "Chip 2"
}
],
"type": "chips"
}
]
]
}
Suggestion chips act like a user text query when the user clicks on it, therefore, you can just create a route that can be triggered by text of the chip and get the text query from the webhook request sent to your webhook to return the respective information. For example:
Intent:
Route:
Then in your webhook, you can get the parameter value in the text field of the webhook request which you will refer to in order to create a webhook response with the respective information.
Here’s an example in Node.js using Express:
app.post("/webhook", (req, res) => {
let option = req.body.text;
let jsonResponse = {
fulfillment_response: {
messages: [
{
text: {
//fulfillment text response to be sent to the agent
text: [`You've chosen the ${option} option`]
}
}
]
}
};
res.json(jsonResponse);
});
Alternatively, you can also use entity types and assign the selected chip into a parameter that will be also sent to your webhook.
To assign the text of the chip to a parameter, the intent of the route should contain training phrases that are annotated to an entity type containing all of the options. For example:
Intent:
Entity Type:
Then in your webhook, you can get the parameter value in the intentInfo.parameters.parameter_id.resolvedValue field of the webhook request which you will refer to in order to create a webhook response with the respective information.
Here’s an example in Node.js using Express:
app.post("/webhook", (req, res) => {
let option = req.body.intentInfo.parameters.options.resolvedValue;
let jsonResponse = {
fulfillment_response: {
messages: [
{
text: {
//fulfillment text response to be sent to the agent
text: [`You've chosen the ${option} option`]
}
}
]
}
};
res.json(jsonResponse);
});
Results:
There is a simple albeit hacky way I have discover possible (tested in es). Which is to make a chip and get its element then force clicking it
We can listen to button click and I detect that it was empty button with just text. Then I use renderCustomCard to make a chip. Everything inside dialogflow messenger are hidden deep inside nested shadowRoot. But as of now its structure allow us to get the chip out to call click() on it. In effect it make it seem very the same as user manually click the chip
const dfMessenger = document.querySelector('df-messenger');
dfMessenger.addEventListener('df-button-clicked',function(event) {
if(event.detail.element.event || event.detail.element.link)
return;
dfMessenger.renderCustomCard([
{
"type": "chips",
"options": [
{
"text": event.detail.element.text
}
]
}
]);
var messageList = dfMessenger.shadowRoot.querySelector("df-messenger-chat").shadowRoot.querySelector("df-message-list").shadowRoot;
var chips = [...messageList.querySelectorAll("df-chips")].flatMap((chips) => [...chips.shadowRoot.querySelectorAll(".df-chips-wrapper>a")]).filter((a) => a.innerHTML.indexOf(event.detail.element.text) > -1);
if(chips.length > 0)
chips.slice(-1)[0].click();
});
Working for today. No guarantee they will block this method in the future. But I actually guess they would implement actual postback button in similar manner later after beta version

How to create Adaptive card and continue dialog on specific response in azure bot node v4

I am currently trying to create a waterfall that starts with an adaptive card.
Originally I had the waterfall working with ChoicePrompt on every step, but on step 1 I wanted 2 choices to openUrl - so changed to an adaptive card to start (is this required or is there a way to openUrl from a specific response in ChoicePrompt the user gives?)
The issue here is that every response (that isn't the openUrl buttons) leads to the adaptive card to repeat itself rather than pass the non-openUrl choice to the next step of the dialog.
I am also storing each dialog response (currently in an array, which I clear at the end of the dialog) to perform a certain action based on all responses combined (is there a better way to save user responses than pushing into an array?)
var answers = [];
async firstStep(stepContext) {
var send = {
text: 'question',
attachments: [
{
"contentType": "application/vnd.microsoft.card.hero",
"content": {
"text": null,
"buttons": [
{
"type": "imBack",
"title": "one",
"value": "one"
},
{
"type": "openUrl",
"title": "two",
"value": "https://example.com"
},
{
"type": "openUrl",
"title": "three",
"value": "https://example.com"
},
]
}
}
]
}
return await stepContext.context.sendActivity(send);
}
async secondStep(stepContext) {
const resp = stepContext.result.value;
answers.push(resp);
return await stepContext.prompt('ChoicePrompt', {
prompt: questions[1],
choices: ChoiceFactory.toChoices(options[1]),
style: ListStyle.suggestedAction
});
}
async thirdStep(stepContext) {
const resp = stepContext.result.value;
answers.push(resp);
return await stepContext.prompt('ChoicePrompt', {
prompt: questions[2],
choices: ChoiceFactory.toChoices(options[2]),
style: ListStyle.suggestedAction
});
}
async finalStep(stepContext) {
const resp = stepContext.result.value;
answers.push(resp);
// get func
var fun = await this.func(answers);
//do stuff with what function returns
// reset quiz
answers = [];
return await stepContext.endDialog();
}
So to summarise, the initial adaptive card, I would like it to continue to repeat itself if anything other than one is returned, but if one is returned I would like it to move onto the next part of the dialog with that value, and save that value (maybe better than I do above).
Lastly, if there is an easy way to openUrl in the current tab and not a new one that would be great.
Any insight here on how to work with hero cards would be really helpful.
Thanks in advance.

Suggestions are not working in actions on google

I create a chat bot using dialogflow and actions on google library. In the back-end code I have created function including if else. I added new suggestions in the "else if". That are not display in the actions on google simulator.Another suggestions are working. Only that time it is not displayed. Please give me some instructions for fixed that.
this is my code:
'use strict';
const functions = require('firebase-functions');
const {
dialogflow,
Suggestions,
Carousel,
Image,
Table,
List,
} = require('actions-on-google');
const app = dialogflow({debug: true});
// Constants for list and carousel selection
const Order_Food = 'order food';
const Extra_Product = 'extra product';
const Spa_Reservation = 'spa reservation';
const Restaurant_Booking = 'restaurant booking';
app.intent('user.provide_room_number', (conv) => {
conv.ask('Great! I can help you with the following. Please select
from the options below.');
//conv.ask(new Suggestions('Order Food', 'Extra Product',
'Restaurant', 'Spa'));
// Create a carousel
conv.ask(new Carousel({
items: {
// Add the first item to the carousel
[Order_Food]: {
synonyms: [
'order food',
'food',
],
title: 'Food',
description: 'Can order some food',
image: new Image({
url: 'http://www.restauranteelpalacete.com/wp-content/uploads/2018/01/Online-Food-Ordering.jpg',
alt: 'Food',
}),
},
// Add third item to the carousel
[Spa_Reservation]: {
synonyms: [
'spa',
'spa reservation',
],
title: 'Spa Reservation',
description: 'Can put the reservation on the spa.',
image: new Image({
url: 'https://res.klook.com/images/fl_lossy.progressive,q_65/c_fill,w_1295,h_720,f_auto/w_80,x_15,y_15,g_south_west,l_klook_water/activities/kykzulvt1t71kwhnmkik/OasisSpa.jpg',
alt: 'Spa',
}),
},
// Add fourth item to the carousel
[Restaurant_Booking]: {
synonyms: [
'restaurant',
'restaurant booking',
],
title: 'Restaurant',
description: 'Can put the reservation on the Restaurant.',
image: new Image({
url: 'https://cdn-image.foodandwine.com/sites/default/files/1501607996/opentable-scenic-restaurants-marine-room-FT-BLOG0818.jpg',
alt: 'Restaurant',
}),
},
},
}));
});
app.intent('actions_intent_OPTION-handler', (conv, params, option) => {
// Get the user's selection
// Compare the user's selections to each of the item's keys
if (!option) {
conv.ask('You did not select any item from the list or carousel');
} else if (option === 'order food') {
conv.ask(new SimpleResponse({speech:"Absolutely, have you decided what you like or you could go through the in room dinning menu. \n Do you need order Food.?",text:"Absolutely, have you decided what you like or you could go through the in room dinning menu. Do you need order Food.?"}));
conv.ask(new Suggestions(["Ok", "Done", "Thanks"]));
} else if (option === 'spa reservation') {
conv.ask(new Suggestions('Yes I need Spa.'));
conv.ask(`We have an excellent Spa that offer exquisite treatment packages. You can select one of the options. We have quite a few free slots today. Do you need information about that.`);
} else if (option === 'restaurant booking') {
conv.ask(`We have some dining options for you today. Do you want more information. `);
conv.ask(new Suggestions('I need restaurant.'));
} else {
conv.ask('You selected an unknown item from the list, or carousel');
}
});
You should design your conversation bot more efficiently.In second elseif condition, after a suggestion no further conversation should happen in an intent.Suggestion are for triggering intent. For your case best scenario would be to create a follow up intent.
Suggestion is not appearing in conversation because, in every rich response at least one simple response should be present. Try below code. Hope this helps you.
conv.ask(new SimpleResponse({speech:"Absolutely, have you decided what you like or you could go through the in room dinning menu. \n Do you need order Food.?",text:"Absolutely, have you decided what you like or you could go through the in room dinning menu. Do you need order Food.?"}));
conv.ask(new Suggestions(["Ok", "Done", "Thanks"]));

Sending multiple embeds in one message

How do you send multiple embeds in one message? Sending multiple like this:
await message.channel.send({embed: { //Send a new embed
title: "Embed 1",
fields: [{
name: "Description",
value: "The Description"
}]
}},
embed: { //Send a new embed
title: "Embed 2",
fields: [{
name: "Description",
value: "The Description"
}]
}});
Gives an output of:
[object Object]
Embed 2
Description: The Description
I cant find any documentation on sending multiple embeds, there is some mention of it in the discord.js file, sending a list of embeds. Though I have tried this and it also doesn't work.
It certainly is possible by using a Webhook to send your message!
Here is the documentation for the WebhookMessageOptions, as you can see the embeds option accepts an array of MessageEmbed.
Quick example:
message.channel.createWebhook('Webhook Name', message.author.displayAvatarURL)
.then(w => w.send({embeds: [
new Discord.MessageEmbed().setAuthor('Embed 1'),
new Discord.MessageEmbed().setAuthor('Embed 2'),
]}));
This works for up to 10 embeds.
Try using Richembed, it's easier to editing and the style is better.
You need to add two or how much embeds you want to send, like this:
let bot1embed = new Discord.RichEmbed()
.setAuthor("Test Bot")
.setThumbnail(client.user.displayAvatarURL)
.setColor("#00ff00")
.addField("Hello!", "Hello World")
.addField("I'm an bot", "I'm a bot");
message.channel.send(bot1embed);
let bot2embed = new Discord.RichEmbed()
.setAuthor("Test Bot")
.setThumbnail(client.user.displayAvatarURL)
.setColor("#00ff00")
.addField("Hello!", "Hello World")
.addField("I'm an bot", "I'm a bot");
message.channel.send(bot2embed);
This way the bot will send two embeds when someone used the command.

chrome extension context menus, how to display a menu item only when there is no selection?

What I want to do is:
if the user does not select anything, display menu item A;
if the user selects something, display menu item B.
So far what I can get is:
if the user does not select anything, display menu item A;
if the user selects something, display both A and B.
I want to know:
how to make item A disappear when there is selection?
Many thanks!
Below is my code:
var all = chrome.contextMenus.create
({
"title": "A",
"contexts":["page"],
"onclick": doA
});
var selection = chrome.contextMenus.create
({
"title": "B",
"contexts":["selection"],
"onclick": doB
});
You would need to inject a content script to every page which would check on mousedown event (before menu is displayed) whether or not there is a selection on the page, and then would send a command to a background page to create according menu items.
content_script.js:
document.addEventListener("mousedown", function(event){
//right click
if(event.button == 2) {
if(window.getSelection().toString()) {
chrome.extension.sendRequest({cmd: "createSelectionMenu"});
} else {
chrome.extension.sendRequest({cmd: "createRegularMenu"});
}
}
}, true);
background.html
chrome.extension.onRequest.addListener(function(request) {
if(request.cmd == "createSelectionMenu") {
chrome.contextMenus.removeAll(function() {
chrome.contextMenus.create({
"title": "B",
"contexts":["selection"],
"onclick": doB
});
});
} else if(request.cmd == "createRegularMenu") {
chrome.contextMenus.removeAll(function() {
chrome.contextMenus.create({
"title": "A",
"contexts":["page"],
"onclick": doA
});
});
}
});
Perhaps this was a bug or the functionality was different back when the OP originally asked the question (jun 17, 2011); but today, the OP's code works exactly as expected by simply using contexts parameters in her chrome.contextMenus.create() calls, exactly as she has it written.

Resources