I'm editing the Cloud Functions for Firebase on the Fulfillment page on DialogFlow. I'm trying to respond to an Intent with an audio file playback. Specifically I'm targeting the Telephony integration.
I understand that a text message like
<speak><audio src="https://actions.google.com/sounds/v1/alarms/bugle_tune.ogg"></audio></speak>
Should play the audio.
But what is the interface to send it back so it would work?
Just using agent.add() doesn't seem to work (it reads the SSML string outloud)
You cannot use the client API library to do this at this time. You'll need to craft the JSON response yourself. Please see my answer here which should be helpful: DIalogflow Telephony integration is interpreting SSML response from webhook as normal text
Basically do the same but use TelephonyPlayAudio instead of TelephonySynthesizeSpeech.
Related
I am using Dialogflow and fullfilment for dynamic response and integration has been done with Hangout. Text response is working fine. But when i use rich media like CARDS (Hangout API), It is not working. Can you please let me know what was i am missing or how to use cards for hangouts using dialogflow-fullfilment agent?
Stack Driver Log Image
Thanks and Regards,
Ramchandra-Sah GANESH
I wrote a blog about this, since I've figured some small quirks.
The best is to test this first within the Dialogflow UI console, by choosing custom payloads for Hangouts.
Note:
The first key can't be called cards, but it has to be named to hangouts
This hangouts key points to an object, not an array (of cards)
Have a look into this blog to get further details. (For example on using this with webhook code) How to build chatbots for Hangouts with Dialogflow by using custom payloads and cards.
Good evening!
Everything is configured with whatsapp.
I use auto reply app and dialogFlow to make a little robot that automatically responds to certain things already ordered on whatsapp. The problem is that I can only use text as an answer; I would like from dialogFlow to respond with an image. Maybe should I use custom payload?
I'm building a chatbot with Telegram and Google Dialogflow. When I integrate Dialogflow into Telegram via the Dialogflow "Integrations" tab, Dialogflow registers as a Webbhook service to intercept all received Telegram messages. Unfortunately, I cannot act on non-text messages like stickers, images and Telegram payment checkout messages.
I wish Dialogflow would forward all the received messages to my custom fulfillment webhook in the Fallback intent. In my webhook, I would be able to parse the received body and act on the unsupported telegram messages accordingly.
I know that there is a FACEBOOK_MEDIA event in Dialogflow. This event is fired when an image is sent via a Facebook messenger to Dialogflow. Is there something like a TELEGRAM_MEDIA or even a TELEGRAM_UNSUPPORTED event available?
I would really like to be able to act on more than just text messages in Dialogflow.
Do I need to add another layer between Telegram and Dialogflow to be able to act on all Telegram messages?
What I have analyzed so far:
I forward all Google Dialogflow interaction logs to Google Cloud stackdriver. It seems as if non-text messages are completely ignored by Dialogflow. I see no log entry at all when I send a non-text message to my telegram bot. On the other hand, when I send a text, the log and its JSON payload appears immediately in the stackdriver.
I created a fallback intent in Dialogflow and enabled "Enable webhook call for this intent" in the Fulfillment section. Unfortunately, my fulfillment function doesn't fire on any non-text events. Dialogflow blocks it in the first place.
For future reference, I will try to give a possible, but not an out-of-the-box solution to my own question. If someone has a better solution, this would be great.
It seems to me, that at the moment of writing this, a full interaction experience is only possible by implementing an intermediate layer between Telegram and Dialogflow. Telegram might contact this intermediate layer, which processes the received message and uses the Dialogflow SDK/API for the Google NLP goodness. For the (not yet) supported Telegram messages this intermediate layer must implement its own logic.
Currently, there seems to be no way to handle non-telegram text messages via dialogflow, however there is an open report about it here if it may interest https://issuetracker.google.com/issues/157202508
I want my google assistance app to read response using custom user voice. I'm using webhook to send responses to user queries. Currently, I'm sending text responses. I have built a custom voice model using (https://lyrebird.ai/). When my webhook is triggered I would like to first convert the text response to audio using my custom model and send the output audio to google home. Is this possible? Or Is there any better ways of achieving this.
Yes you can synthesize your own audio and return it as Ssml in an audio tag:
https://developers.google.com/actions/reference/ssml
Can Dialogflow respond with Video? I wished to have a video triggered as an intent response but I think there is no way out within the selection.
Through custom payload. You can attach a video type response like in the case of facebook messenger
This is not possible. According to the documentation, the intent response could be a static text or Webhook response, which is a JSON. You may consider to provide a link of the video in the response so the user can get it by clicking it.