Dialogflow - Google Assistant: setFollowupEvent after showing a message - node.js

Here it is my use case: somewhere in my fulfillment, while handling the Intent A, I want to call Intent B using setFollowupEvent. Something like:
function intentA_Handler(){
....
agent.add('This is the response when calling Intent A');
agent.setFollowupEvent('call_intent_B_event');
}
function intentB_Handler(){
agent.add('I am in intent B now.');
....
}
What I'm expecting:
The assistant Shows & Speaks out the string This is the response when calling Intent A
Then calling agent.setFollowupEvent('call_intent_B_event'); and Showing & Speaking out the string I am in intent B now.
What happens:
The assistant immediately shows & speaks out to me the string I am in intent B now and omits the first string This is the response when calling Intent A
Already tried:
function intentA_Handler(){
....
new Promise((resolve, reject) => {
agent.add('This is the response when calling Intent A');
resolve();
});
agent.setFollowupEvent('call_intent_B_event');
}
But still the same result. Any idea how to achieve my goal?

This is the expected behavior. The followup event goes back to Dialogflow and not the Actions on Google platform. The second intent response will go back to the assistant as the only response coming from the agent.
If you use the Actions on Google client library (https://developers.google.com/actions/reference/nodejsv2/overview), the it has a way to pass parameters with the follow up event: https://actions-on-google.github.io/actions-on-google-nodejs/classes/dialogflow.dialogflowconversation.html#followup
You could use the parameter values to track additional information you want to include in the final response.

In general - you don't need to use followup events if you are doing things through fulfillment. Remember - Intents usually represent the user doing or saying something, not what you want the reply to be.
If you need to call some code in both the Intent Handler for A and for B - just call that code as another function. If you want to conclude with the same message in both A and B - then you can just call add() with the messages you want in both A and B - either directly or by calling a common function. Perhaps something like this:
function addMessageB(){
agent.add('This was from adding message B.');
}
function intentA_Handler(){
....
agent.add('This is the response when calling Intent A');
addMessageB();
}
function intentB_Handler(){
addMessageB();
....
}
There are limitations (you can only add() two Simple Messages, for example), but this would be the general approach.

Related

DialogFlow if statement conv.ask in app.intent

I want to run this code in dialogflow ,basically i want to use if condition
for example.
if my intent "Done" reply response [Text Response] =>'Now add 2 more and say got it' then
it switch to my another intent called "alright" so the response should be "you no. is 1" and end the converstation.
similary
if my intent "Done" reply response [Text Response] =>'Now add 4 more and say got it' then
it switch to my another intent called "alright" so the response should be "you no. is 2" and end the converstation.
and goes on.
just look at the SS Below
enter image description here
enter image description here
'use strict';
// Import the Dialogflow module from the Actions on Google client library.
const {dialogflow} = require('actions-on-google');
// Import the firebase-functions package for deployment.
const functions = require('firebase-functions');
// Instantiate the Dialogflow client.
const app = dialogflow({debug: true});
var numr=0;
app.intent('done',(conv,output)=>{
if(conv.ask ==='Now add 2 more, and say got it'){
numr=1;
return numr;
}
else{
numr=2;
return numr;
}
});
app.intent('Alright',(conv)=>{
conv.close('your number is '+ numr);
});
// Set the DialogflowApp object to handle the HTTPS POST request.
exports.dialogflowFirebaseFulfillment = functions.https.onRequest(app);
There are a number of issues with your code that make it difficult to understand exactly what you're doing. However, there are a few things to note. I'm assuming you're using JavaScript here.
You have a syntax error in how you are using else. The else statement should be followed by an execution block. You look like you're following it with a condition. Perhaps you are trying to use else if there?
conv.ask is a function. You're then assigning a string to it which would remove it as a function. I don't think this is what you mean to do here.
In the Intent Handler for the "done" Intent, you're not sending anything back to the user using the conv.ask() function since you just removed that as a function.
Setting a global variable does not guarantee the value of it will be preserved in between Intent Handler calls. If you want a value saved during a conversation, you should use a Context or conv.data or other means.

How can I save data in storage from the response to an external API

We are using firebase cloud functions for our fulfillment and an external rest api for our crud operations.
We have one intent with a few followups and it looks like this
- Create Note
- - Create Note - fallback*
- - - Create Note - fallback - yes
- - - Create Note - fallback - no
* the fallback allows us to capture free text
In our webhook I have the following fulfillment
app.intent("Create Note", (conv) => {
if (!conv.user.storage.note) conv.user.storage.note = {};
conv.ask("Go ahead, talk to Talkatoo!");
});
app.intent("Create Note - fallback", (conv) => {
conv.user.storage.input = conv.input.raw;
let response = conv.user.storage.note
? conv.user.storage.note.body.concat(conv.input.raw)
: conv.user.storage.input;
conv.ask("So far this is what I heard you say, let me know if this is complete. ", response);
});
app.intent("Create Note - fallback - yes", (conv) => {
// based on the conv.user.storage.note object
// either make a call to create a note or update a note
// make call to external api and based on the response
// set the value for conv.user.storage.note
conv.ask("Great news, let me save that for you!");
});
app.intent("Create Note - fallback - no", (conv) => {
// based on the conv.user.storage.note object
// either make a call to create a note or update a note
// make call to external api and based on the response
// set the value for conv.user.storage.note
// Send the user back to Create Note to capture the rest of their input
conv.followup("my_custom_event");
});
The issue is that conv.user.storage.note is getting set when I get the response from the API, but then it is getting reset to empty and so a new note is created each time. I'm trying to append the various inputs from the user to be one note
Based on ...
app.intent("Create Note - fallback - yes", (conv) => {
// based on the conv.user.storage.note object
// either make a call to create a note or update a note
// make call to external api and based on the response
// set the value for conv.user.storage.note
conv.ask("Great news, let me save that for you!");
});
it looks like either you haven't written the code for the API call yet, or you removed that code before you posted on StackOverflow. Is that correct?
Can you update your question to show where and how you send an HTTP request to your external REST API?
For anyone who this might end up helping, I found a post and realized that dealing with responses to async calls might need more thought (to work in work arounds) How to make asynchronous calls from external services to actions on google?
For our use case we didn't need the response data ie, we didn't need to communicate any of the response data to the user, so instead we saved the user's multiple inputs in user.storage and when they were complete we save the full response.
app.intent("Intent - fallback", (conv) => {
conv.user.storage.input = conv.user.storage.input
? conv.user.storage.input.concat(conv.input.raw)
: conv.input.raw;
conv.ask("Is that complete or would you like to continue?");
});
app.intent("Intent - fallback - yes", (conv) => {
createNote(conv); <--- makes call to save
conv.ask("Great news, let me save that for you!");
});
app.intent("Intent - fallback - no", (conv) => {
conv.followup("my_custom_event");
});

'Until loop' analogue needed - in order to continue bot dialog - after some status 'marker' is updated

'Until loop' analogue needed to continuously read status variable from helper function - and then (when the status variable is 'as we need it') - to resume bot conversation flow.
In my bot (botbuilder v.3.15) I did the following:
During one of my dialogues I needed to open external url in order
to collect some information from the user through that url.
After that I posted collected data (with conversation ID and other info) from that url to my bot app.js
file
After that I needed to resume my bot conversation
For that I created helper file - helper.js in which 'marker' variable is 'undefined' when the data from url is not yet collected, and 'marker' variable is some 'string' when the data is collected and we can continue our bot conversation
helper.js
var marker;
module.exports = {
checkAddressStatus: function() {
return marker;
},
saveAddressStatus: function(options) {
marker = options.conversation.id;
}
}
I can successfully update variable 'marker' with my data, by calling saveAddressStatus function from app.js.
However, when I get back to writing my code which is related to bot conversation flow (To the place in code after which I opened url - in file address.js, and from where I plan to continuously check the 'marker' variable whether it is already updated - in order to fire 'next()' command and continue with session.endDialogWithResult -> and then -> to further bot conversation flows - I cannot find the equivalent of 'until loop' in Node.js to resume my session in bot dialog - by returning 'next()' and continuing with the bot flow.
address.js
...
lib.dialog('/', [
function (session, args, next) {
...
next();
},
function (session, results, next) {
// Herocard with a link to external url
// My stupid infinite loop code, I tried various options, with promises etc., but it's all not working as I expect it
while (typeof helper.checkAddressStatus() == 'undefined') {
console.log('Undefined marker in address.js while loop')
}
var markerAddress = helper.checkAddressStatus();
console.log(markerAddress);
next(); // THE MOST IMPORTANT PART OF THE CODE - IF markerAddress is not 'undefined' - make another step in dialog flow to end dialog with result
function(session, results) {
...session.endDialogWithResult({markerAddress: markerAddress})
}
...
Any ideas how to make a very simple 'until loop' analoque in this context - work?
Having your bot stop and wait for a response is considered bad practice. If all of your bot instances are stuck waiting for the user to fill out the external form, your app won't be able to process incoming requests. I would at least recommend adding a timeout if you decide to pursue that route.
Instead of triggering your helper class in the endpoint you created, you should send a proactive message to the user to continue the conversation. To do this, you will need to get the conversation reference from the session and encode it in the URL that you send to the user. You can get the conversation reference from the session - session.message.address - and at the very least you will need to encode the bot id, conversation id, and the serviceUrl in the URL. Then when you send the data collected from the user back to the bot, include the conversation reference details for the proactive message. Finally, when your bot receives the data, recreate the conversation reference and send the proactive message to the user.
Here is how your conversation reference should be structured:
const conversationReference = {
bot: {id: req.body.botId },
conversation: {id: req.body.conversationId},
serviceUrl: req.body.serviceUrl
};
Here is an example of sending a proactive message:
function sendProactiveMessage(conversationReference ) {
var msg = new builder.Message().address(conversationReference );
msg.text('Hello, this is a notification');
msg.textLocale('en-US');
bot.send(msg);
}
For more information about sending proactive messages, checkout these samples and this documentation on proactive messages.
Hope this helps!

Handle audio play completion callback in dialogflow (Media responses)

I'm handling an intent by playing a MediaObject. I want to create an intent handler that will catch the callback of media play completion, the documentation shows an example on how to write fulfillment code to handle it.
Building your fulfillment
The code snippet below shows how you might write the fulfillment code
for your Action. If you're using Dialogflow, replace
actions.intent.MEDIA_STATUS with the action name specified in the
intent which receives the actions_intent_MEDIA_STATUS event, (for
example, "media.status.update“).
I am confused with the part of the dialogflow instructions. The intent that I handle and return a MediaObject is called smoothie-02 and I have a fallback for it which what gets handled after the media finishes being played, but I want to create another intent and handle it there instead. What I want to do is to create a new intent that would handle it, instead of it going to the fallback intent of smoothie-02 intent.
smoothie-02 handler
app.dialogFlow.intent('smoothie-02', (conv) => {
const welcomeContext = getConvContext(conv, AppContexts.WELCOME);
givenName = welcomeContext.parameters['given-name'];
fruitTypes = welcomeContext.parameters['FruitTypes'];
if (!conv.surface.capabilities.has('actions.capability.MEDIA_RESPONSE_AUDIO')) {
conv.ask('Sorry, this device does not support audio playback.');
return;
}
conv.contexts.set("actions_capability_media_response_audio", 5);
// Zoe says something
let response = `Ooh good choice ${givenName} ! `;
response += fruitTypes.length > 1 ? `${fruitTypes[0]} and ${fruitTypes[1]}` : `${fruitTypes[0]} `;
response += `${drinkType} ` ;
response += 'coming right up. But will you first turn me on?';
console.log(response);
conv.ask(response);
conv.ask(new Suggestions("Don't be shy"));
// Blender plays
conv.ask(new MediaObject({
name: 'Blender Sound',
url: 'https://storage.googleapis.com/zoe-mortimer.appspot.com/blender.wav',
}));
});
What I needed to do is create a new intent and add actions_intent_MEDIA_STATUS in the Events, and that will be the intent that would handle the media playback finished callback. Credits to this article!

dialogflow fullfilment and firebase response time

I am trying to build a simple chatbot with DialogFlow.
My aim is to give information from user question, like : where can I slackline above water in croatia ? I have two parameters (croatia, waterline) and a list of slackline places.
So I need a data base to retrieve information from parameters. DialogFlow allows fulfillment with Firebase. I build a database with places (name, country, type of slack) and enable webhook call for my intent.
I use Inline Editor and index.js
const parameters = request.body.queryResult.parameters;
var country = parameters.country.toString();
function show(snap) {
console.log('snap');
agent.add(JSON.stringify(snap.val(),null,2));
}
function slkplc(agent) {
var testRef;
firebase.database().ref('slackplace').once('value',show);
}
// Run the proper function handler based on the matched Dialogflow intent name
let intentMap = new Map();
intentMap.set('slack place', slkplc);
agent.handleRequest(intentMap);
But I do not get the expected result while trying it on DialogFlow or Google Assistant. The function show is asynchronously called but too late and the response is not available for DialogFlow :
I see three way to deal with this problem :
use blocking call to database : another database ?
treat asynchronous message with DialogFlow ???
response to user that an error occured.
The third that I choose, but it is always on error.
After trying several things to wait data from database response, the only thing I managed is to freeze the response, therefore the timeout of DialogFlow - 5s -and Firebase - 60s - were reached.
A workaround
Another way to do it is to separate database acquisition and request/response from DialogFlow. The data of database is collected outside of the dialogflowFirebaseFulfillment
var data;
var inidata = firebase.database().ref().on('value',function(snap) {
console.log('snap');
data = snap.val();
});
exports.dialogflowFirebaseFulfillment = functions.https.onRequest((request, response) => {
const agent = new WebhookClient({ request, response });
...
function slkplc(agent) {
agent.add(JSON.stringify(data,null,2));
}
// Run the proper function handler based on the matched Dialogflow intent name
let intentMap = new Map();
intentMap.set('slack place', slkplc);
agent.handleRequest(intentMap);
}
Now I can do what I want with data, and I am able to find the place where I can practice waterline in croatia. But there is always something weird, the data of the database is duplicated ...
The "right" solution is option 2 that you suggest: since you're doing an asynchronous call, you need to handle this correctly when working with the dialogflow-fulfillment library.
Basically, if your handler makes an asynchronous call, it needs to be asynchronous as well. To indicate to the handleRequest() method that your handler is async, you need to return a Promise object.
Firebase's once() method returns a Promise if you don't pass it a callback function. You can take advantage of this, return that Promise, and also handle what you want it to do as part of a .then() clause. It might look something like this:
function slkplc(agent) {
var testRef;
return firebase.database().ref('slackplace').once('value')
.then( snap => {
var val = snap.val();
return agent.add( JSON.stringify( val, null, 2 ) );
});
}
The important part isn't just that you use a Promise, but also that you return that Promise.

Resources