I'm building a multi-language-educational app on DialogFlow.
and i hit a big problem : Switch recognition language.
Any suggestions how could i achieve that or any other approaches?
I have added to DialogFlow web-interface secondary language.
I'm using action-on-google sdk with node.js webhook.
UPDATE :
let responseToUser = {
//fulfillmentMessages: richResponsesV2, // Optional, uncomment to enable
//outputContexts: [{ 'name': `${session}/contexts/weather`, 'lifespanCount': 2, 'parameters': {'city': 'Rome'} }], // Optional, uncomment to enable
fulfillmentText: 'This is from Dialogflow\'s Cloud Functions for Firebase editor! :-)', // displayed response
payload: {
audioConfig : {
"audioEncoding": 'AUDIO_ENCODING_FLAC',
"sampleRateHertz": 16000,
"languageCode": 'de',
"phraseHints": ["gutten tag"]
}
},
function sendResponse (responseToUser) {
// if the response is a string send it as a response to the user
if (typeof responseToUser === 'string') {
let responseJson = {fulfillmentText: responseToUser}; // displayed response
response.json(responseJson); // Send response to Dialogflow
} else {
// If the response to the user includes rich responses or contexts send them to Dialogflow
let responseJson = {};
// Define the text response
responseJson.fulfillmentText = responseToUser.fulfillmentText;
responseJson.payload = responseToUser.payload;
// Send the response to Dialogflow
console.log('Response to Dialogflow: ' + JSON.stringify(responseJson));
response.json(responseJson);
The request json :
{
"responseId": "c4ea35d3-1455-4142-b7d2-22417c5880ae",
"queryResult": {
"queryText": "hi",
"action": "default",
"parameters": {},
"allRequiredParamsPresent": true,
"fulfillmentText": "This is from Dialogflow's Cloud Functions for Firebase editor! :-)",
"fulfillmentMessages": [
{
"text": {
"text": [
"This is from Dialogflow's Cloud Functions for Firebase editor! :-)"
]
}
}
],
"webhookPayload": {
"sampleRateHertz": 16000,
"languageCode": "de",
"phraseHints": [
"gutten tag"
],
"audioEncoding": "AUDIO_ENCODING_FLAC"
},
"outputContexts": [
{
"name": "projects/homehf-master/agent/sessions/824b34b9-45f8-4c65-9377-a31242d3414b/contexts/test_context",
"lifespanCount": 5
}
],
"intent": {
"name": "projects/homehf-master/agent/intents/0259f134-6d4d-4aed-920d-9240adbe38fe",
"displayName": "starter_intent"
},
"intentDetectionConfidence": 0.75,
"diagnosticInfo": {
"webhook_latency_ms": 44
},
"languageCode": "en"
},
"webhookStatus": {
"message": "Webhook execution successful"
}
}
Dialogflow requires a language with the audio request, if you'd like to change that language you'll need to issue a new request. You can implement some logic in your webhook and frontend to try the request again with a different language.
To do this add some data in the payload field of your webhook response response indicating that you'd like to issues a new request with the a different language specified. In your client, check the data in the response payload and if the data indicates retrying with another language, send another request with a different language specified in the input audio config with the same audio.
Related
I am building a bot so whenever a specific task is initiated I want to send an image to the user.
exports.handler = function(context, event, callback) {
let response = {
"actions": [
{
"show": {
"body": "Twilio Owls",
"images": [{
"label": "Original Owl",
"url": "https://demo.twilio.com/owl.png"
}]
}
}]
}
callback(null, response)
}
I tried using the above code inside my function and I have linked it to my Twilio autopilot task, It is working in the simulator, but when I am testing it on Whatsapp Image does not appear in the Whatsapp chat only body get displayed.
I tested and received the owl image. I noticed have a rogue {, maybe that is it?
exports.handler = function(context, event, callback) {
let response = {
"actions": [
{
"show": {
"body": "Twilio Owls",
"images": [{
"label": "Original Owl",
"url": "https://demo.twilio.com/owl.png"
}]
}
}]
}
callback(null, response)
};
I'm using dialogflow-fulfillment for fulfillment. I am trying to send a payload using the following function;
function getName(agent) {
let options = {
"options": [
{
"id": "make_reservation",
"text": "Reservations",
"payload": "make_reservation"
}
]
};
let newPayload = new Payload('PLATFORM_UNSPECIFIED', options);
agent.add('added by webhook');
agent.add(newPayload);
}
But I'm getting a null node in the fulfillment response.
{
"payload": {
"null": {
"options": [
{
"id": "make_reservation",
"text": "Reservations",
"payload": "make_reservation"
}
]
}
},
"outputContexts": []
}
In the place of the null node, I need the options node. Any idea how to do that?
Thanks
After doing some more search, I got the answer! Payload takes three parameters.
new Payload(platform, payload, options);
where the default options are {sendAsMessage: false, rawPayload: false}. Since rawPayload is false, payload will be nested on the platform name. In my case platform was PLATFORM_UNSPECIFIED, that's why it was nested on a null node.
So the solution is to set rawPayload to true.
let newPayload = new Payload('PLATFORM_UNSPECIFIED', options, {rawPayload: true});
const payload = {
payload: {
google: {
expectUserResponse: true,
richResponse: {
items: [
{
simpleResponse: {
ssml: 'hi i am vamsi',
}
}
]
},
},
},
};
I'm using above payload for sending a response to google assistant.
I want to know payload to send as the default response because this response only displays on google assistant. I need a default to display for every platform.
The webhook response can be as simple as
{
"fulfillmentText": "Hi I am Vamsi"
}
Is it possible to test your Dialogflow fulfillment webhook locally using the cloud functions emulator, and if so, how should I format the request?
I've read through all the documentation I can find, including the guide at https://firebase.google.com/docs/functions/local-emulator, and of particular interest was this previous question which seems to hit on a similar point:
Unit test Actions on Google Dialogflow locally
I am able to invoke my fulfillment function with the functions shell, however no matter how I attempt to format the body I can only ever seem to trigger my fallback intent or the error catching intent.
I can verify on the Actions on Google simulator that my webhook successfully responds with the default welcome intent when given the input "hello", but when using the same request JSON data as input to my function locally, I am directed to the fallback intent.
Is it the case that the functions emulator cannot perform the proper intent-matching locally and therefore always triggers the fallback intent, or am I simply not formatting my request right? Any help would be greatly appreciated!
Here is the invocation format which I am using, and the response from the shell:
firebase > fulfillment({method: 'POST',json: true,body:
require("project/collabrec/testData.json")});
Sent request to function.
firebase > info: User function triggered, starting execution
info: Fallback intent triggered.
info: Execution took 15 ms, user function completed successfully
RESPONSE RECEIVED FROM FUNCTION: 200, {
"payload": {
"google": {
"expectUserResponse": true,
"richResponse": {
"items": [
{
"simpleResponse": {
"textToSpeech": "I didn't quite catch that. Could you say that again?"
}
}
]
}
}
}
}
Here is the testData.json content:
{
"user": {
"userId": "ABwppHFR0lfRsG_UM3NkvAptIkD2iUpIUNxFt-ia05PFuPajV6kRQKXu_H_ECMMe0lP_WcCsK64sH2MEIg8eqA",
"locale": "en-US",
"lastSeen": "2018-10-19T15:20:12Z"
},
"conversation": {
"conversationId": "ABwppHHerN4CIsBZiWg7M3Tq6NwlTWkfN-_zLIIOBcKbeaz4ruymv-nZ4TKr6ExzDv1tOzszsfcgXikgqRJ9gg",
"type": "ACTIVE",
"conversationToken": "[]"
},
"inputs": [
{
"intent": "actions.intent.TEXT",
"rawInputs": [
{
"inputType": "KEYBOARD",
"query": "hello"
}
],
"arguments": [
{
"name": "text",
"rawText": "hello",
"textValue": "hello"
}
]
}
],
"surface": {
"capabilities": [
{
"name": "actions.capability.MEDIA_RESPONSE_AUDIO"
},
{
"name": "actions.capability.SCREEN_OUTPUT"
},
{
"name": "actions.capability.AUDIO_OUTPUT"
},
{
"name": "actions.capability.WEB_BROWSER"
}
]
},
"isInSandbox": true,
"availableSurfaces": [
{
"capabilities": [
{
"name": "actions.capability.SCREEN_OUTPUT"
},
{
"name": "actions.capability.AUDIO_OUTPUT"
},
{
"name": "actions.capability.WEB_BROWSER"
}
]
}
],
"requestType": "SIMULATOR"
}
And here is my cloud function webhook:
const {dialogflow, Image} = require('actions-on-google');
const admin = require('firebase-admin');
const functions = require('firebase-functions');
const app = dialogflow();
app.catch((conv, error) => {
console.log("Error intent triggered.")
console.error(error);
conv.ask('Sorry, I ran into an error. Please try that again.');
});
app.fallback((conv) => {
console.log("Fallback intent triggered.")
conv.ask("I didn't quite catch that. Could you say that again?");
})
app.intent('Default Welcome Intent', (conv) => {
console.log("Welcome intent triggered.")
conv.ask("Welcome!!");
});
exports.fulfillment = functions.region('europe-west1').https.onRequest(app);
Using Node v8.1.4, and package versions:
"#google-cloud/common-grpc": "^0.9.0",
"#google-cloud/firestore": "^0.17.0",
"#google-cloud/functions-emulator": "^1.0.0-beta.5",
"actions-on-google": "^2.4.1",
"firebase-admin": "^6.0.0",
"firebase-functions": "^2.0.5"
The issue is that you're using the JSON that comes from the AoG Simulator, but this shows the JSON that AoG is sending to Dialogflow. Dialogflow processes this and sends your webhook a different JSON which includes the results of processing the AoG JSON and determining the intent, parameters, and other information.
What you are doing should work - if you have the Dialogflow JSON. You have a couple of ways to do this:
The most straighforward is to run your webhook on a place that can receive the POST from Dialogflow and look at the conv.request object, which should be able to give you the JSON you need.
If you're running the webhook on a local dev machine (as you suggest you are), I tend to start up an ngrok tunnel. The tunnel gives a public HTTPS server, which is very useful, and has the side effect of giving me a console that I can use to see exactly the contents of the request and response JSON.
Finally, you should be able to go into the project settings in Dialogflow and turn on Cloud Logging. The log includes the request that is sent to your webhook along with the response that you get from it.
I am generating my first bot working with node.js and heroku but finding some difficulties to understand the persistent menu functionalities.
Question 1) How do can I attach event as callbacks?
function persistentMenu(sender){
request({
url: 'https://graph.facebook.com/v2.6/me/thread_settings',
qs: {access_token:token},
method: 'POST',
json:{
setting_type : "call_to_actions",
thread_state : "existing_thread",
call_to_actions:[
{
type:"postback",
title:"FAQ",
payload:"DEVELOPER_DEFINED_PAYLOAD_FOR_HELP"
},
{
type:"postback",
title:"I Prodotti in offerta",
payload:"DEVELOPER_DEFINED_PAYLOAD_FOR_HELP"
},
{
type:"web_url",
title:"View Website",
url:"https://google.com/"
}
]
}
}, function(error, response, body) {
console.log(response)
if (error) {
console.log('Error sending messages: ', error)
} else if (response.body.error) {
console.log('Error: ', response.body.error)
}
})
}
Question 2) The only way I have found for empty the persistent menu and generating a new one is with a delete request via terminal ("as Facebook documented")m is there a possibily to clear inserting a refresh function on my app.js file?
curl -X DELETE -H "Content-Type: application/json" -d '{"setting_type":"call_to_actions","thread_state":"existing_thread"}' "https://graph.facebook.com/v2.6/me/thread_settingsaccess_token=PAGE_ACCESS_TOKEN"
The FB example robot is not well structured for call backs. I haven't found a good way to structure the example in Node callback or promise model. I'm sure a Node expert can reorg it.
As for the persistent menu, if you send an empty call_to_actions array the menu will disappear. The menu seems a bit 'sticky' however as it does not immediately appear/disappear when the message is sent.
I incorporated your snippet into my example robot. You can see it at
https://messenger.com/t/dynamicmemorysolutions
The source is at:
https://github.com/matthewericfisher/fb-robot
See the add/remove menu commands and functions.
EDIT: The persistent menu API has been updated. See this question for more details.
this worked for me:
function menuButton() {
var messageData = {
setting_type : "call_to_actions",
composerinputdisabled :"TRUE",
thread_state : "existing_thread",
call_to_actions:[
{
type:"postback",
title:"¿Tiempo de espera?",
payload:"ACTUALIZAR"
},
{
type:"postback",
title:"Ver Promociones",
payload:"DEVELOPER_DEFINED_PAYLOAD_FOR_START_ORDER"
}
]
}
request({
uri: 'https://graph.facebook.com/v2.6/me/thread_settings',
qs: { access_token: PAGE_ACCESS_TOKEN },
method: 'POST',
json: messageData
}, function (error, response, body) {
if (!error && response.statusCode == 200) {
var recipientId = body.recipient_id;
var messageId = body.message_id;
console.log("Successfully sent generic message with id %s to recipient %s",
messageId, recipientId);
} else {
console.error("Unable to send message.");
console.error(response);
console.error(error);
}
});
}
And I call this function at the beggining
app.post('/webhook', function(req, res){
var data = req.body;
if (data.object == 'page') {
menuButton();
data.entry.forEach(function(entry) {
var pageID = entry.id;
var timeOfEvent = entry.time;
// Iterate over each messaging event
entry.messaging.forEach(function(event) {
if (event.message) {
receivedMessage(event);
}else if (event.postback) {
receivedPostback(event);
} else {
console.log("Webhook received unknown event: ", event);
}
});
});
res.sendStatus(200);
}
})
What I have not being able to do is to remove the option of free text input. Facebook claimed now is possible yet have found no instructions or examples on how to do it. Any clues?
If you want to disable the free text input, you shoud add the following parameter to your persistent menu request:
"composer_input_disabled": true
and not
composerinputdisabled :"TRUE"
The FB API document states that the API link to hit for applying persistent menu to the page specific bot is:
https://graph.facebook.com/v2.6/me/messenger_profile?access_token=<PAGE_ACCESS_TOKEN>
Notice the me after version number i.e v2.6 in this specific case. However, this did not worked for a lot of people
There is small change in the API link to hit:
graph.facebook.com/v2.6/Page ID/messenger_profile?access_token=PAGE ACCESS TOKEN
Notice that me is replaced with the fb Page Id.
And the sample payload can still be the same:
{
"get_started": {
"payload": "Get started"
},
"persistent_menu": [
{
"locale": "default",
"composer_input_disabled": false,
"call_to_actions": [
{
"title": "Subscribe",
"type": "postback",
"payload": "subscribe"
},
{
"title": "Stop notifications",
"type": "nested",
"call_to_actions": [
{
"title": "For 1 week",
"type": "postback",
"payload": "For_1_week"
},
{
"title": "For 1 month",
"type": "postback",
"payload": "For_1_month"
},
{
"title": "For 1 year",
"type": "postback",
"payload": "For_1_year"
}
]
},
{
"title": "More",
"type": "nested",
"call_to_actions": [
{
"title": "Fresh jobs",
"type": "postback",
"payload": "fresh jobs"
},
{
"title": "Like us",
"type": "web_url",
"url": "https://www.facebook.com/onlysoftwarejobs/"
},
{
"title": "Feedback",
"type": "web_url",
"url": "https://docs.google.com/forms/d/e/1FAIpQLScjgFRbfBLznO55kFIskcH_eFc23zRSUUxzIgv_o44uj0GMpw/viewform"
}
]
}
]
}
]
}
Notice that it is mandatory to configure get_started button before setting up the persistent_menu.