Dialogflow assistant app exiting after successful fulfillment - webhooks

I have a dialogflow assistant app with 3 intents. The first intent asks the user for location and name details from google. I am using a webhook for the fulfillment of this intent. I am able to extract the user information name and location, but after it is showing output from webhook, it is exiting from flow. But it is supposed to pass the location parameters to next intent and stay on the flow. Can anybody help me how to stop assistant from exiting?
Here is the webhook code
'use strict';
const functions = require('firebase-functions');
const DialogflowApp = require('actions-on-google').DialogflowApp;
exports.dialogflowFirebaseFulfillment = functions.https.onRequest((request, response) => {
const requestPermission = (app) => {
app.askForPermissions('To report ', [app.SupportedPermissions.NAME, app.SupportedPermissions.DEVICE_PRECISE_LOCATION]);
};
const userInfo = (app) => {
if (app.isPermissionGranted()) {
const address = app.getDeviceLocation().coordinates;
const name = app.getUserName().givenName;
if (name) {
app.tell(`You are name ${name}`);
}
else {
// Note: Currently, precise locaton only returns lat/lng coordinates on phones and lat/lng coordinates
// and a geocoded address on voice-activated speakers.
// Coarse location only works on voice-activated speakers.
app.tell('Sorry, I could not figure out where you are.Plaese try again');
}
} else {
app.tell('Sorry, I could not figure out where you are.Please try again');
}
};
const app = new DialogflowApp({request, response});
const actions = new Map();
actions.set('request_permission', requestPermission);
actions.set('user_info', userInfo);
app.handleRequest(actions);
});

The problem is that you are calling app.tell() in your code which is a signal to the Assistant to send the message and then end the conversation.
If you want to send the message and then leave the microphone open for the user to reply, you should use app.ask() instead. It takes the same parameters - the only difference is that it expects the user to reply.
So that portion of your code might look something like
if (name) {
app.ask(`You are name ${name}. What would you like to do now?`);
}
(You should make sure that the prompt for the user is one that they will expect to reply. The review process will reject your Action if you reply and it isn't obvious that the user is supposed to reply to you.)

Related

DialogFlow with Telegram: How to receive an image and save it along with the conversation

I'm developing a chat bot for Telegram using DialogFlow, but I can't go through two topics, and I can't find the documentation for them.
The flow of the conversation, is the user answer some closed questions and send an image.
How do I get this image?
And to save her along with the other answers?
The answers need to be saved as a form/survey and not as a conversation history.
I have a similar setup in my chatbot. I store the answers in a Firebase database.
In order to interact with the Firestore Database you should implement a Fulfillment
You can see a guide on how to implement Firebase for DialogFlow here
Here you can see a sample of my code. In general lines after setting up the connection to the Firebase database you just want to map your intents to your functions using intentMap.set.
As you said you are using closed answers you can set intets to handle the responses and each "final" intent will trigger a different function that will write a different message to the db.
To write the response to the Firesbase database you just only need to implement admin.database().ref().push().set({}) with the information and the desired structure.
In my example I also store the conversation Id from the chat payload and the date.
'use strict';
const functions = require('firebase-functions');
const admin = require('firebase-admin');
const {WebhookClient} = require('dialogflow-fulfillment');
const {Card, Suggestion} = require('dialogflow-fulfillment');
//const DialogflowApp = require('actions-on-google').DialogflowApp;
process.env.DEBUG = 'dialogflow:debug'; // enables lib debugging statements
admin.initializeApp({
credential : admin.credential.applicationDefault(),
databaseURL: 'ws://YOURDATABASE.firebaseio.com/'
});
exports.dialogflowFirebaseFulfillment = functions.https.onRequest((request, response) => {
const agent = new WebhookClient({ request, response });
console.log('Dialogflow Request headers: ' + JSON.stringify(request.headers));
console.log('Dialogflow Request body: ' + JSON.stringify(request.body));
var userId;
let conv = agent.conv();
const ROOTREF = admin.database().ref();
const actions = new Map();
let intentMap = new Map();
intentMap.set('Default Fallback Intent', fallback);
intentMap.set('NoTunel', handleWriteToDbNoTunnel(agent));
agent.handleRequest(intentMap);
function assignConv(agent){
userId = agent.parameters.UserId;
return admin.database().ref('Users/'+ userId).set({
Apellido:"XXXX",
Nombre:"XXXX",
chatId:333,
});}
function fallback(agent) {
agent.add(`I didn't understand`);
agent.add(`I'm sorry, can you try again?`);
}
var token = "YOUR TOKEN HERE";
var url = "https://api.telegram.org/bot"+ token;
function handleWriteToDbNoTunnel(agent){
const Dia = new Date();
if(matricula !== "")
return admin.database().ref('Limpieza/').push().set({
chatId: request.body.queryResult.outputContexts[3].parameters.telegram_chat_id+'"',
Field1: answer1,
Field2: answer2,
day: day.getTime()
});
}
});
Also if you want to store images with the user responses you can implement the getfile method from the telegram api and store the image code or the image itself
I am adding this answer to slightly improve on Chris32's answer.
There is a better way to get the value of the Telegram Chat ID as I am using it in a personal project.
I will go end to end to explain my approach.
I have mapped some files to some specific intents. In my intent-mapper.js file, I have mapped Default Welcome Intent to welcome.js file as prescribed in the documentation for the Dialogflow Fufillment library for NodeJS (Please note that the library is deprecated and not being updated, personally I am using a fork of the repo that I have worked on personally).
const intentMap = new Map();
intentMap.set('Default Welcome Intent', welcome);
.
.
Then, in welcome.js,
const globalParameters = {
'name': 'global-parameters',
'lifespan': 9999,
'parameters': {}
};
globalParameters.parameters.telegramChatId = agent.originalRequest?.payload?.data?.chat?.id || -1;
.
.
agent.setContext(globalParameters);
The telegramChatId variable in the global parameters context will save the value for the chat ID which can be passed to a helper function to send a message. In order to to retrieve the value from the global parameters, the code snippet is this.
const globalParameters = agent.getContext('global-parameters');
const telegramChatId = globalParameters.parameters.telegramChatId;
Then the Telegram message helper function is largely the same as in Chris32's answer. The message can be any string and chatId can be passed as an argument to the following helper function.
const TelegramBot = require('node-telegram-bot-api');
const { telegramBotToken } = process.env.TELEGRAM_BOT_TOKEN;
const bot = new TelegramBot(telegramBotToken, { polling: false });
const sendTelegramTextMessage = (message, chatId) => {
try {
bot.sendMessage(chatId, message, {parse_mode: 'html'});
} catch (err) {
console.log('Something went wrong when trying to send a Telegram notification', err);//remove console.log()
}
};
The reason I have put this all in a context since in my use case I am sending push notifications via Telegram once the user asks for it (this happens later in the conversation flow), so I have implemented it this way. The main point to note is that the agent object already has the detectIntentRequest variable saved inside it which in turn has the value we need as a part of its payload. Here's a snippet of the same.
Please note I have removed many lines from my code for brevity, but in a nutshell, the chat ID can be accessed from
agent.originalRequest?.payload?.data?.chat?.id
And the value for the telegram bot token is a secret value which can be saved in an environment variable or Secrets Manager. Please note my answer explains a better way to retrieve the chat ID without needing to refer directly to the request object since Dialogflow Fulfillment library already caches the value in the body for us. The other stuff for receiving and sending images is explained in the main answer.

Google Action asking for parameters before account linking

I am using OAuth Authorisation Flow for my google action and for some reason, it is asking for parameters then initiating account linking and then asking for parameters again.
Example Response:
Code
app.intent('Create Channel Intent', async (conv, params) => {
if (!conv.user.access.token) {
conv.ask(new SignIn());
} else {
var locale = conv.user.locale;
if (locale === 'hi-IN') {
var accessToken = conv.user.access.token;
var channelNameRaw = params.channelname;
var channelNameData = await helperFunctions.hinditranslate(channelNameRaw);
var channelNameLwr = channelNameData.toLowerCase();
var channelName = helperFunctions.replaceWhitespacesFunc(channelNameLwr);
const headers = await helperFunctions.login(accessToken);
const speechText = await helperFunctions.createChannel(channelName, headers);
conv.ask(speechText);
} else {
var accessToken = conv.user.access.token;
var channelNameRaw = params.channelname;
var channelNameData = channelNameRaw.toLowerCase();
var channelName = helperFunctions.replaceWhitespacesFunc(channelNameData);
const headers = await helperFunctions.login(accessToken);
const speechText = await helperFunctions.createChannel(channelName, headers);
conv.ask(speechText);
}
}
});
Dialogflow
You shouldn't add training phrase to your SignIn event intent. If the intent which asks for signin permission has training phares would be enough. Because it directs to actions_intent_SIGN_IN event and continues on there unless user didn't signed in yet. I guess you made these two action in one intent so this makes it confused and tries to call same intent and asks for parameters.
If you had to use this way try to use context so you can pass parameter values to this intent on second call.
I am %99 sure that your intent's sloth filling is off.
Your Create Channel Intent has channelname as required parameter. Without Sloth-filling, intents won't call your server/code until every required parameter is fulfilled.
What happening is:
You are calling Create Channel intent and it asks for channel name as it is mandatory
User gives a channel name, intent calls your code as all required parameters are fulfilled.
Your code trigger sign_in intent as user haven't signed in yet.
User gives permission which triggers actions_intent_SIGN_IN event
Your Create Channel Intent has been called as it has actions_intent_SIGN_IN as trigger and asks for channelname as this intent is brand new.
To fix do one of this:
Enable sloth-filling at the bottom of intent.
Add an output context and add #[CONTEXT-NAME].channelname as default value to channelname parameter (without bracelets) . You can assign default values by clicking the appearing 3 dot when your mouse is over the parameter.
Split Capturing sign_in event from your Create Channel Intent
Hope it helps.

How to get device coarse location for each webhook request in actions on google

Hi i'm trying to get device coarse location by getting permission from user.But i required to get device location details for each webhook request without asking permission again and again.So I'm bit confused how to do this one.
Here is the below code which i tried.
const {Permission} = require('actions-on-google');
const {WebhookClient} = require('dialogflow-fulfillment');
const agent = new WebhookClient({ request: req, response: res });
function x(agent){
conv.ask(new Permission({context:'To Locate You',permissions:'DEVICE_COARSE_LOCATION'}));
}
function userinfo(agent){
var conv=agent.conv();
var resp=conv.arguments.get('PERMISSION');
console.log(conv.device.location);
if(resp){
var country=conv.device.location.country;
var speech="you are located in "+country;
conv.ask(speech);
agent.add(conv);
}else{
conv.ask('Sorry, I could not figure out where you are');
agent.add(conv);
}
}
Please check the helper functions here. You need to do the following:
Create an intent to ask for permission.
In that intent, ask the permission you want
Create second intent to capture user's response to intent by putting the Dialogflow event actions_intent_PERMISSION to that intent.
In the webhook handle of the second intent check for confirmation.
Ask Permission in First Intent
app.intent('FIRST_INTENT_NAME', (conv) => {
// Choose one or more supported permissions to request:
// NAME, DEVICE_PRECISE_LOCATION, DEVICE_COARSE_LOCATION
const options = {
context: 'To address you by name and know your location',
// Ask for more than one permission. User can authorize all or none.
permissions: ['NAME', 'DEVICE_PRECISE_LOCATION'],
};
conv.ask(new Permission(options));
});
Capture result in Second Intent
app.intent('SECOND_INTENT_NAME', (conv, params, confirmationGranted) => {
const {name} = conv.user;
if (confirmationGranted) {
if (name) {
conv.ask(`I'll send the driver you're way now ${name.display}.`);
}
}
});
For exact understanding with code example, check out this GitHubb example link.

Set context on Google Actions through Dialogflow fulfillment

I'm using the Dialogflow editor fulfillment to build a conversation on Google Assistant, but I'm not sure how to use the agent.setContext function.
When I ask "Turn off my device" the first step returns successfully. The problem happens when the user responds with "TV", for example. The system returns with another context, ignoring the one I set.
When the user directly asks "Turn off my TV in the kitchen" the system also works perfectly. Because of it I think that the entities are correctly defined.
Conversation 1 (success):
Turn off my TV in the kitchen
Right. Turning off.
Conversation 2 (fail):
Turn off my device //Gives the same intent that Conversation 1
OK, which device?
TV
"My bot" isn't responding right now. Try again soon. //It throws "'final_response' must be set". I think it's because of generic intent actions.intent.TEXT.
My code:
The code below is nested in exports.dialogflowFirebaseFulfillment = functions.https.onRequest and the intent is called by intentMap.set('smarthome.device.switch.off', setDeviceOff);
const agent = new WebhookClient({ request: request, response: response });
function setDeviceOff(agent) {
const device = agent.parameters.device;
const room = agent.parameters.room;
const context= 'device-switch'; //I tried 'smarthome.device.switch.off' too
let resp = commandDevice(device, room, 'Off', agent, context);
return resp;
}
function commandDevice(device, room, cmd, agent, context) {
var conv = agent.conv();
if(device===''){
conv.ask("OK, which device?");
}else if (room===''){
conv.ask("Got it. of what room?");
}else{
conv.ask("Right. Turning off.");
}
agent.setContext({
name:context,
lifespan: 5, //Tried 0, 4 and 3 too
parameters:{
'device':device,
'room':room
}
});
agent.add(conv);
return true;
}
So I tried another version and the same problem persists:
const app = dialogflow();
app.intent('welcome', conv =>{
conv.ask("Welcome to auto obari. What can I do for you?");
});
app.intent('Default Fallback Intent',conv =>{
conv.ask("I didn't understand, can you try again?");
});
app.intent('smarthome.device.switch.off', (conv,{device, room})=> {
const context= 'device-switch';
if(device===''){
conv.ask("OK, which device?");
}else if (room===''){
conv.ask("Got it. of what room?");
}else{
conv.ask("Right. Turning off.");
}
const parameters = {'device':device, 'room': room};
conv.contexts.set(context, 5, parameters);
});
exports.dialogflowFirebaseFulfillment = functions.https.onRequest(app); //I tried exports.factsAboutGoogle, but it threw a error too.
The contexts are the same, but the intent is different.
If using conv, you may also try like this:
app.intent('<INTENT>', conv => {
conv.ask('<RESPONSE>');
const parameters = {'param1':param1, 'param2': param2}};
conv.contexts.set('welcome-context', 5, parameters);
});
and access it like here :
const conv.contexts.get(<context>).parameters[<param>];
you can try this method too it worked for me
const AppContexts = {CONTEXT_ONE: 'context-one'};
const app = dialogflow();
In current intent (for output context)
conv.contexts.set(AppContexts.CONTEXT_ONE, 1);
In next intent (for input context )
const context = conv.contexts.get(AppContexts.CONTEXT_ONE);

dialogflow Webhookclient "request_" property

I am trying to build up a facebook messenger chatbot using Dialogflow. In the dialogflow fulfillment inline editor, I found that I can use agent.request_.body to get the body of the request. I assume "request_" is a property of WebhoodClient object? But I couldn't find any documentation elaborate that, could you please advise if my understanding is correct and where I can find the reference or documentation?
const agent = new WebhookClient({ request, response });
console.log(JSON.stringify(agent.request_.body));
Thanks
Google provides documentation for Dialogflow webhooks here, which include this sample webhook to inspect parameters and dynamically create slot filling prompts:
exports.dialogflowFirebaseFulfillment = functions.https.onRequest((request, response) => {
const agent = new WebhookClient({ request, response });
function flight(agent) {
const city = agent.parameters['geo-city'];
const time = agent.parameters['time'];
const gotCity = city.length > 0;
const gotTime = time.length > 0;
if(gotCity && gotTime) {
agent.add(`Nice, you want to fly to ${city} at ${time}.`);
} else if (gotCity && !gotTime) {
agent.add('Let me know which time you want to fly');
} else if (gotTime && !gotCity) {
agent.add('Let me know which city you want to fly to');
} else {
agent.add('Let me know which city and time you want to fly');
}
}
let intentMap = new Map();
intentMap.set('flight', flight);
agent.handleRequest(intentMap);
});
My guess would be to add
console.log(agent);
right before defining the flight function, then checking the logs to see which objects agent contains, then adding iterations of console.log(agent.fakeObjectName) until you find the information you're looking for.
If you're following the deployment process recommended in Actions on Google's Codelabs level 2, your logs will show up in the Firebase console, like this:
Hope that helps!
Just a note.
I had a code similar to this:
const city = agent.parameters['geo-city'];
There is an icon that suggest it's better written in dot notation.
that is gone after I changed it to:
const city = agent.parameters.geo-city;

Resources