Unable to get device coarse location when testing with google mini device - dialogflow-es

Hi I'm facing an issue that, i'm unable to get device coarse location when i tested on google mini device with same dialogflow account.I'm able to get user's name but not device location. I'm frustrated regarding this issue.Please help to resolve this.
Here is my code,
const express = require('express');
const bodyParser = require('body-parser');
const {dialogflow,Permission} = require('actions-on-google');
const app = dialogflow({
clientId : 'xyz'
});
app.intent('Default Welcome Intent', conv => {
const options = {
context: 'To locate you',
permissions: ['NAME','DEVICE_COARSE_LOCATION'],
};
conv.ask(new Permission(options));
});
app.intent('User_info',(conv, params, permissionGranted) => {
console.log(conv);
var location=conv.device.location;
console.log(location);
if(permissionGranted){
const name = conv.user.name;
console.log(name);
var resp="you are located at"+conv.device.location.city;
conv.ask(resp);
}else{
conv.close("sorry I'm unable to locate you right now. Okay bye now");
}
});
const expressApp=express().use(bodyParser.json())
expressApp.post('/',app);
expressApp.listen(3000);

I just tested the DEVICE_COARSE_LOCATION permission, and found an empty object in my Firebase logs where I expected to see the location.
Have you tried using DEVICE_PRECISE_LOCATION instead? Here's a sample how that works:
// Handle the Dialogflow intent named 'Default Welcome Intent'.
app.intent('Default Welcome Intent', (conv) => {
conv.ask(new Permission({
context: 'Hi there, to get to know you better',
permissions: ['NAME', 'DEVICE_PRECISE_LOCATION']
}));
});
// Handle the Dialogflow intent named 'actions_intent_PERMISSION'. If user
// agreed to PERMISSION prompt, then boolean value 'permissionGranted' is true.
app.intent('actions_intent_PERMISSION', (conv, params, permissionGranted) => {
if (!permissionGranted) {
conv.ask(`Ok, no worries. What's your favorite color?`);
conv.ask(new Suggestions('Blue', 'Red', 'Green'));
} else {
console.log(conv);
conv.data.userName = conv.user.name.given;
conv.data.userLatitude = conv.device.location.coordinates.latitude;
conv.data.userLongitude = conv.device.location.coordinates.longitude;
conv.ask(`Thanks, ${conv.data.userName} at latitude ${conv.data.userLatitude} and longitude ${conv.data.userLongitude}. What's your favorite color?`);
}
});

Related

How to create the intent via code in dialogflow?

I am beginner in dialogflow and trying to create intent and context through code, but i am unable to achieve it.This is the code i am using,
i referred the below link also but no help,
https://cloud.google.com/dialogflow/es/docs/how/manage-intents#create_intent
but getting below error:
function main(
projectId = 'trainer-gulw',
displayName = 'intent_001',
trainingPhrasesParts = [
'Hello, What is weather today?',
'How is the weather today?',
],
messageTexts = ['Rainy', 'Sunny']
) {
const dialogflow = require('#google-cloud/dialogflow');
// Instantiates the Intent Client
const intentsClient = new dialogflow.IntentsClient();
async function createIntent() {
// Construct request
// The path to identify the agent that owns the created intent.
const agentPath = intentsClient.projectAgentPath(projectId);
const trainingPhrases = [];
trainingPhrasesParts.forEach(trainingPhrasesPart => {
const part = {
text: trainingPhrasesPart,
};
// Here we create a new training phrase for each provided part.
const trainingPhrase = {
type: 'EXAMPLE',
parts: [part],
};
trainingPhrases.push(trainingPhrase);
});
const messageText = {
text: messageTexts,
};
const message = {
text: messageText,
};
const intent = {
displayName: displayName,
trainingPhrases: trainingPhrases,
messages: [message],
};
const createIntentRequest = {
parent: agentPath,
intent: intent,
};
// Create the intent
const [response] = await intentsClient.createIntent(createIntentRequest);
console.log(`Intent ${response.name} created`);
}
createIntent();
// [END dialogflow_create_intent]
}
TIA
From the screen shot, it looks like you're running this on your local machine. The error message suggests that the environment variables that point to your credentials isn't setup.
See the Before you Begin section of the library documentation for how to setup that environment. One of the steps includes setting up authentication by downloading credentials and setting an environment variable to point to them. This will also set the project ID you're working with.

Google Dialogflow doesn't utilize my changes

I'm trying to create a simple AI chatbot command for my discord bot and it's working out, however, my Dialogflow doesn't seem to utilize any changes that I made like new intents or different responses. it always just returns the text from before the changes or just doesn't return anything at all.
I might be really stupid.
This is my code:
const Discord = require("discord.js")
const axios = require('axios');
const uuid = require('uuid');
const dialogflow = require('#google-cloud/dialogflow');
require("dotenv").config();
const projectId = "mydialogprojectid";
console.log(process.env.GOOGLE_APPLICATION_CREDENTIALS)
const config = require(`../config.json`)
exports.run = async(client, message, args) => {
message.channel.startTyping();
// A unique identifier for the given session
const sessionId = uuid.v4();
console.log(sessionId)
// Create a new session
const sessionClient = new dialogflow.SessionsClient();
const sessionPath = sessionClient.projectAgentSessionPath(projectId, sessionId);
// The text query request.
const request = {
session: sessionPath,
queryInput: {
text: {
// The query to send to the dialogflow agent
text: args.join(' '),
// The language used by the client (en-US)
languageCode: 'en-US',
},
},
};
// Send request and log result
const responses = await sessionClient.detectIntent(request);
const result = responses[0].queryResult;
if (result.intent) {
console.log(` Intent: ${result.intent.displayName}`);
} else {
console.log(` No intent matched.`);
}
console.log(result)
message.channel.stopTyping();
return message.channel.send(result.fulfillmentText ? result.fulfillmentText : "Something went wrong, forgive me please! I'm still in beta.")
}
I have the same issue it works on the Dialogflow console etc but when using it in discord it wouldnt find the intent etc like you described.
At first i thought this was just some cache issue. I will update this if i find a solution.

TEAMS bot. "TypeError: source.on is not a function" in 'createConversation' method

I have a TEAMS node.js bot running locally (with ngrok). I receive messages from TEAMS client and echo works
context.sendActivity(`You said '${context.activity.text}'`);
I need to send a proactive message, but i get an error creating conversation pbject.
My code:
...
await BotConnector.MicrosoftAppCredentials.trustServiceUrl(sServiceUrl);
var credentials = new BotConnector.MicrosoftAppCredentials({
appId: "XXXXXXXXXXXX",
appPassword: "YYYYYYYYYYYYY"
});
var connectorClient = new BotConnector.ConnectorClient(credentials, { baseUri: sServiceUrl });
const parameters = {
members: [{ id: sUserId }],
isGroup: false,
channelData:
{
tenant: {
id: sTenantId
}
}
};
// Here I get the error: "TypeError: source.on is not a function"
var conversationResource = await connectorClient.conversations.createConversation(parameters);
await connectorClient.conversations.sendToConversation(conversationResource.id, {
type: "message",
from: { id: credentials.appId },
recipient: { id: sUserId },
text: 'This a message from Bot Connector Client (NodeJS)'
});
String values are correct and I have a valid connectorClient.
Thanks in advance,
Diego
I think you are close but just need to make a couple modifications. It's hard to tell because I can't see all of your code.
Here, I add the Teams middleware to the adapter in index.js and am making the call that triggers the proactive message from within a waterfall step in mainDialog.js. The location of the code shouldn't matter as long as you can pass context in.
Also, with regards to your message construction, sending the text alone is sufficient. The recipient is already specified in the paremeters with the rest of the activity properties assigned via the connector. There is no need to reconstruct the activity (unless you really need to).
Lastly, be sure to trust the serviceUrl. I do this via the onTurn method in mainBot.js.
Hope of help!
index.js
const teams = require('botbuilder-teams');
const adapter = new BotFrameworkAdapter({
appId: process.env.MicrosoftAppId,
appPassword: process.env.MicrosoftAppPassword
})
.use(new teams.TeamsMiddleware())
mainDialog.js
const { ConnectorClient, MicrosoftAppCredentials } = require('botframework-connector');
async teamsProactiveMessage ( stepContext ) {
const credentials = new MicrosoftAppCredentials(process.env.MicrosoftAppId, process.env.MicrosoftAppPassword);
const connector = new ConnectorClient(credentials, { baseUri: stepContext.context.activity.serviceUrl });
const roster = await connector.conversations.getConversationMembers(stepContext.context.activity.conversation.id);
const parameters = {
members: [
{
id: roster[0].id
}
],
channelData: {
tenant: {
id: roster[0].tenantId
}
}
};
const conversationResource = await connector.conversations.createConversation(parameters);
const message = MessageFactory.text('This is a proactive message');
await connector.conversations.sendToConversation(conversationResource.id, message);
return stepContext.next();
}
mainBot.js
this.onTurn(async (context, next) => {
if (context.activity.channelId === 'msteams') {
MicrosoftAppCredentials.trustServiceUrl(context.activity.serviceUrl);
}
await next();
});
I created my bot with Yeoman, so I had a running example which can respond to an incoming message.
https://learn.microsoft.com/es-es/azure/bot-service/javascript/bot-builder-javascript-quickstart?view=azure-bot-service-4.0
With that, I can respond to an incoming message and it works
My complete code:
index.js: // Created automatically, not modified
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.
const dotenv = require('dotenv');
const path = require('path');
const restify = require('restify');
// Import required bot services.
// See https://aka.ms/bot-services to learn more about the different parts of a bot.
const { BotFrameworkAdapter } = require('botbuilder');
// This bot's main dialog.
const { MyBot } = require('./bot');
// Import required bot configuration.
const ENV_FILE = path.join(__dirname, '.env');
dotenv.config({ path: ENV_FILE });
// Create HTTP server
const server = restify.createServer();
server.listen(process.env.port || process.env.PORT || 3978, () => {
console.log(`\n${ server.name } listening to ${ server.url }`);
console.log(`\nGet Bot Framework Emulator: https://aka.ms/botframework-emulator`);
console.log(`\nTo test your bot, see: https://aka.ms/debug-with-emulator`);
});
// Create adapter.
// See https://aka.ms/about-bot-adapter to learn more about how bots work.
const adapter = new BotFrameworkAdapter({
appId: process.env.MicrosoftAppId,
appPassword: process.env.MicrosoftAppPassword
});
// Catch-all for errors.
adapter.onTurnError = async (context, error) => {
// This check writes out errors to console log .vs. app insights.
console.error(`\n [onTurnError]: ${ error }`);
// Send a message to the user
await context.sendActivity(`Oops. Something went wrong!`);
};
// Create the main dialog.
const myBot = new MyBot();
// Listen for incoming requests.
server.post('/api/messages', (req, res) => {
adapter.processActivity(req, res, async (context) => {
// Route to main dialog.
await myBot.run(context);
});
});
My bot file. bot.js // created automatically, modified to add proactive messages
const { ConnectorClient, MicrosoftAppCredentials, BotConnector } = require('botframework-connector');
const { ActivityHandler } = require('botbuilder');
class MyBot extends ActivityHandler {
constructor() {
super();
// See https://aka.ms/about-bot-activity-message to learn more about the message and other activity types.
this.onMessage(async (context, next) => {
//await context.sendActivity(`You said '${context.activity.text}'`); // It works
var activity = context.activity;
await MicrosoftAppCredentials.trustServiceUrl(activity.serviceUrl);
var connectorClient = new ConnectorClient(credentials, { baseUri: activity.serviceUrl });
const parameters = {
members: [{ id: activity.from.id }],
isGroup: false,
channelData:
{
tenant: {
id: activity.conversation.tenantId
}
}
};
var conversationResource = await connectorClient.conversations.createConversation(parameters);
});
}
module.exports.MyBot = MyBot;
In 'createConversation' is where I get the error:
[onTurnError]: TypeError: source.on is not a function
This code will be in a method, to call it when required, I put it in 'onMessage' just to simplify
I think my code is like yours... But I'm afraid I am missing something or doing sth wrong...
Thanks for your help,
Diego
I have a stack trace of the error, and it seems to be related to 'delayed stream' node module. I add as a response since it is too long for a comment:
(node:28248) UnhandledPromiseRejectionWarning: TypeError: source.on is not a function
at Function.DelayedStream.create ([Bot Project Path]\node_modules\delayed-stream\lib\delayed_stream.js:33:10)
at FormData.CombinedStream.append ([Bot Project Path]\node_modules\combined-stream\lib\combined_stream.js:45:37)
at FormData.append ([Bot Project Path]\node_modules\form-data\lib\form_data.js:74:3)
at MicrosoftAppCredentials.refreshToken ([Bot Project Path]\node_modules\botframework-connector\lib\auth\microsoftAppCredentials.js:127:20)
at MicrosoftAppCredentials.getToken ([Bot Project Path]\node_modules\botframework-connector\lib\auth\microsoftAppCredentials.js:91:32)
at MicrosoftAppCredentials.signRequest ([Bot Project Path]\node_modules\botframework-connector\lib\auth\microsoftAppCredentials.js:71:38)
at SigningPolicy.signRequest ([Bot Project Path]\node_modules\#azure\ms-rest-js\dist\msRest.node.js:2980:44)
at SigningPolicy.sendRequest ([Bot Project Path]\node_modules\#azure\ms-rest-js\dist\msRest.node.js:2984:21)
at ConnectorClient.ServiceClient.sendRequest ([Bot Project Path]\node_modules\#azure\ms-rest-js\dist\msRest.node.js:3230:29)
at ConnectorClient.ServiceClient.sendOperationRequest ([Bot Project Path]\node_modules\#azure\ms-rest-js\dist\msRest.node.js:3352:27)
I found my problem. I must set credentials/conector/trust in the correct order:
credentials = new MicrosoftAppCredentials(process.env.MicrosoftAppId, process.env.MicrosoftAppPassword);
connectorClient = new ConnectorClient(credentials, { baseUri: activity.serviceUrl });
MicrosoftAppCredentials.trustServiceUrl(activity.serviceUrl);

MalformedResponse when asking for Permission

I am struggling on creating a very basic example asking the user for permission. The setup is
Intents:
"request_permission":
- phrase "where am I?"
- action: "request_permission"
- events: <empty>
"user_info":
- phrase: <empty>
- action: "user_info"
- events: "actions_intent_PERMISSION"
Following the official example using the node.js fulfillment library, I write:
'use strict';
const functions = require('firebase-functions');
const {Permission, DialogflowApp} = require('actions-on-google');
const {WebhookClient} = require('dialogflow-fulfillment');
const {Card, Suggestion, SimpleResponse} = require('dialogflow-fulfillment');
const app = dialogflow({debug: true});
process.env.DEBUG = 'dialogflow:debug'; // enables lib debugging statements
exports.dialogflowFirebaseFulfillment = functions.https.onRequest((request, response) => {
const agent = new WebhookClient({ request, response });
const app = new DialogflowApp({request, response});
function welcome_intent(agent) {agent.add(`welcome intent`); }
function fallback_intent(agent) {agent.add(`fallback intent.`); }
function request_permission(agent){
let conv = agent.conv();
conv.ask(new Permission({
context: 'I need your location',
permissions: 'DEVICE_COARSE_LOCATION'
}))
agent.add(conv);
}
function user_info(agent){
if (app.isPermissionGranted()) {
const zipCode = app.getDeviceLocation().zipCode;
if (zipCode) {
app.tell(`your zip code ise ${zipCode}`);
} else {
app.tell('I cannot tell your zip code.');
}
} else {
app.tell('Without permission I cannot tell the zip code');
}
}
let intentMap = new Map();
intentMap.set('Default Welcome Intent', welcome_intent);
intentMap.set('Default Fallback Intent', fallback_intent);
intentMap.set('request_permission', request_permission);
intentMap.set('user_info', user_info);
agent.handleRequest(intentMap);
});
Still, when starting the simulator and activating my app and then asking "where am I?" I get
MalformedResponse
'final_response' must be set.
And yes, I did check the firebase console logs and the "Fulfillment"-Slider "Enable webhook call for this event" is on.
I wouldn't ask here if there would a V2 compatible example with a very basic permission request. I am aware of the answer in Dialogflow v2 API + Actions v2 API: MalformedResponse 'final_response' must be set

How can I properly prompt a user without having LUIS attempt to recognize the response?

I have a bot configured with a LUIS intent recognizer and although the initial root dialog starts as expected when LUIS recognizes the intent, if I prompt a user from within that dialog, the response the user sends back does not resume the suspended dialog but instead is sent back through the LUIS recognizer and begins a new dialog entirely.
This is my bot setup:
const connector = new builder.ChatConnector({
appId: config.get('bot.id'),
appPassword: config.get('bot.secret')
})
super(connector, (session) => this.help(session))
this.set('storage', new BotStorage())
this.use(builder.Middleware.sendTyping(), new MessageSanitizer())
this.on('conversationUpdate', (message: any) => {
console.log(message)
})
const recognizer = new builder.LuisRecognizer(config.get('bot.model'))
this.recognizer(recognizer)
My dialog setup:
this.dialog('/send', [(session, context, next) => {
const amount = builder.EntityRecognizer.findEntity(context.intent.entities, 'builtin.currency')
const recipient = builder.EntityRecognizer.findEntity(context.intent.entities, 'recipient')
const product = builder.EntityRecognizer.findEntity(context.intent.entities, 'product')
session.send('Sure, I can do that.')
session.beginDialog('/send/product', product ? product.entity : null)
}]).triggerAction({
matches: 'send'
})
this.dialog('/send/product', [(session, query, next) => {
if (query) {
session.dialogData.productQuery = query
next()
} else {
builder.Prompts.text(session, 'What type of product did you want to send?')
}
}, (session, results) => {
if (results && results.response) {
session.dialogData.productQuery = results.response
}
session.sendTyping()
ProductService.search(session.dialogData.productQuery).then(products => {
if (!products.length) {
session.send('Sorry, I couldn\'t find any products by that name.')
session.replaceDialog('/send/product')
}
const attachments = products.map(product => {
const image = builder.CardImage.create(session, product.configuration.image)
const valueLine = `$${product.value.min} - $${product.value.max}`
const card = new builder.HeroCard(session)
.title(product.name)
.images([image])
.text(product.description)
.subtitle(valueLine)
.tap(builder.CardAction.postBack(session, product.id))
return card
})
const message = new builder.Message(session)
.text('Okay, I found the following products. Please select the one you\'d like to send.')
.attachments(attachments)
.attachmentLayout(builder.AttachmentLayout.carousel)
builder.Prompts.text(session, message)
}).catch((err: Error) => {
session.error(err)
})
}, (session, response, next) => {
console.log(response)
}])
At the suggestion of a reply below, I also tried to set up the recognizer as part of an IntentDialog rather than on the bot itself and then map the subsequent dialogs to that root dialog like so:
const recognizer = new builder.LuisRecognizer(config.get('bot.model'))
this.intents = new builder.IntentDialog({ recognizers: [recognizer] })
this.intents.matches('send', '/send')
this.dialog('/', this.intents)
this.dialog('/send', [(session, context, next) => {
const amount = builder.EntityRecognizer.findEntity(context.entities, 'builtin.currency')
const recipient = builder.EntityRecognizer.findEntity(context.entities, 'recipient')
const product = builder.EntityRecognizer.findEntity(context.entities, 'product')
session.send('Sure, I can do that.')
session.beginDialog('/send/product', product ? product.entity : null)
}, (session) => {
console.log('hello')
}])
However, this did not help, I am still getting the following error:
/ - WARN: IntentDialog - no intent handler found for None
Indicating that the response is attempting to match a new intent rather than resume the suspended dialog.
An example interaction with the bot that illustrates this issue:
The 'typing' indicator will just continue indefinitely because its trying to match the 'None' intent but none exists.
A match in the triggerAction will take precedence as every message received by the bot is sent through the routing system. You can customize it, but you can also try using the IntentDialog to isolate some of your flows from it.
More: Botframework No Interruptions from other Intent Dialogs until done with current Intent Dialog
In the latest SDKs, you can achieve this without using IntentDialog. All you have to do is use .onEnabled on the LuisRecognizer without modifying any of your existing dialogs:
var recognizer = new builder.LuisRecognizer(url)
.onEnabled(function (context, callback) {
// LUIS is only ON when there are no tasks pending(e.g. Prompt text)
var enabled = context.dialogStack().length === 0;
callback(null, enabled);
});

Resources