We have a framework that implements chatbot / voice assistant logic for handling complex conversations in the health domain. Everything is implemented on our server side. This gives us full control of how responses are generated.
The channel (such as Alexa or Facebook Messenger cloud) calls our webhook:
When user messages, the platform sends these to our webhook: hashed user id, message text (chat message or transcribed voice)
Our webhook responds with the appropriately structured response, which includes text to be displayed, spoken, possibly choice buttons and some images etc. It also includes a flag whether the current session has finished or user input is expected.
Integrating a new channel involves conversion of the response returned into the form expected by a channel and setting some flags (has voice, has display etc.).
This simple framework has worked so far for Facebook Messenger, Cortana, Alexa (a little bit of hacking was needed to abandon it's intent and slot recognition), our web chatbot.
We wanted to write a thin layer of support for Google Assistant action.
Is there any way of passing all the input from Assistant user intact into a webhook such as the one described above and taking full control of the way responses are generated and the end of conversation is determined?
I'd rather not delve into those cumbersome ways of API.AI of structuring a conversation which seems good for a trivial scenarios such as ordering an Uber but seems very bad for longer conversation.
Since you already have a Natural Language Understanding layer for your system, you don't need API.AI/Dialogflow, and you can skip this layer completely. (The NLU is useful, even for large and extensive conversations, but doesn't make sense in your case where you've already defined the conversation through other means.)
You'll need to use the Actions SDK (sometimes known as actions.json after the configuration file it uses) to define triggering phrases, but after that you'll get all the text that the user says as part of your conversation through a webhook that delivers JSON to you. You'll reply with JSON that contains the text/audio response, images on cards, possibly suggestion chips, etc.
Related
We are implementing our customized chatbot using dialog flow. When user enters any text, our javascript code sends this text to our python server and the server interacts with google dialog flow and server gets complete response. I just have couple of questions as below.
When server gets the response from dialog flow, it will process the
response and sends some response to UI. Do we still need to have
fulfillment enabled as our server is getting response? Basically if
server is interacting with dialog flow and getting response, what is
the use of webhook?
Is there anyway to enforce the dialog flow intents require at least
one of entities? I went through Can I make Dialogflow intents require atleast one of the trained entities? which says to enable webhook fulfillment for that intent and if no entities were provided, re prompt the user for at least one of a list of entities. So in my case, if webhook is not needed, do I need to do it in the server once server receives response or is there anyway dialog flow will automatically enforce the condition with out server taking the responsibility?
In your case, no, you don't need to use webhook fulfillment.
You may still wish to use it, however, if you want to separate business logic (which would be in the webhook) from UI/UX logic (which would be in your python server and in the javascript client). But there is no requirement that you separate things this way.
Similarly, you can use your python code to enforce "at least one of" the parameters matching - you're moving that logic from the webhook into your existing server.
Either way, this is a bit kludgy. One alternative if you have different entity types is to have multiple Intents, one for each possible type, and to mark the parameter as required. This way the Intent will only match if the parameter is provided. If you then need to report each of these Intents as the "same" Intent, you can add that logic to your python code.
I'm back again with a question about NLP. I made my own back-end, which on one side can connect to websites, the Google Assistant and Facebook Messenger, and on the other end to Dialogflow. On the side, is logs interactions and does some other database stuff.
Now, I'm trying to connect this back-end to Alexa. I made a project which calls my endpoint. This project has one intent, which has a paramater which should get the raw user input, send it to my back-end, process it, parse and send the response to get back. I feel like there is not a real way to collect and send the raw user input, so I can process it myself (on Dialogflow) instead of using the Amazon way of mapping intents and such.
I know Dialogflow can export to Alexa, but this is not an option for me. I really hope one of you can point me in the right direction.
I just need a way to collect the raw user input, and respond in an Alexa accepted response format.
For Actions on Google for example, I'm using a Custom Project Action Package.
Thanks a lot in advace!
To accept or get any user input, you can use sys.any in google assistant and AMAZON.SearchQuery in AMAZON ALEXA.
In Alexa, You have to add the carrier phrase to use AMAZON.SearchQuery. You can't combine any other slot with AMAZON.SearchQuery.
So there are also some limitations. I hope this answer will help you.
I'm making a dialogflow agent that will be integrated with various platforms (Facebook messenger, slack and maybe a few others) that will have the basic functions of a informational chatbot.
The agent will be for a specific store and I'm wondering if it's possible to trigger some sort of welcome message once the user enters the geofence (in this case, the store)?
Thanks for the help. I haven't found any documentation for this on dialogflow specifically or anywhere else so anything will be awesome.
Note: I'm am not by any means dead set of dialogflow, if AWS Lex offers something like this and it's better, I will take a look. I'm just a bit more used to dialogflow.
This cannot be achieved just by using either Lex or Dialogflow. Because at the end of the day, you are using them to integrate with Messenger/Slack/Whatsapp and these apps will (for obvious good reasons) not share the user's location information with the bot. You will need a helper app which takes the user's location permission as well and triggers the bots for you.
Keep in mind that channels like Messenger and Whatsapp have restrictions over sending messages willy-nilly to users. Messenger has a '24+1' policy Whatsapp also you can only send free form messages in the 24 hour window. But after that you can send chargeable pre-approved "hsm" message templates.
During our testing, we were unable to complete at least one of the behaviors or actions advertised by your app. Please make sure that a user can complete all core conversational flows listed in your registration information or recommended by your app.
Thank you for submitting your assistant app for review!
During testing, your app was unable to complete a function detailed in the app’s description. The reviewer interacted with the app by saying: “how many iphones were sold in the UK?” and app replied “I didn't get that. Can you try with other question?" and left conversation.
How can I resolve the above point to approve my Google Assistant action skills?
Without seeing the code in question or the intent you think should be handling this in Dialogflow, it is pretty difficult - but we can generalize.
It sounds like you have two issues:
Your fallback intent that generated the "I didn't get that" message is closing the conversation. This means that either the "close conversation" checkbox is checked in Dialogflow, you're using the app.tell() method when you should be using app.ask() instead, or the JSON you're sending back has close conversation set to true.
You don't have an intent to handle the question about how many iPhones were sold in the UK. This could be because you just don't list anything like that as a sample phrase, or the two parameters (the one for object type and the one for location) aren't using entity types that would match.
It means that somewhere, either in your app description or in a Dialogflow intent(they have full access to see what's in your intents) you hinted that “how many iphones were sold in the UK?” would be a valid question. Try changing the description/intents to properly match the restrictions of your app.
I have a Facebook Messenger bot (written in NodeJS) and a separate control panel where a user can manage the information that the bot is working with (like inventory stock, etc.). One of those things is a log of all conversations between the bot and a visitor. The control panel allows the admin users to send messages to visitors through the bot. There is an input box where they can type in a message and when they click 'Send', the message goes to the bot app, which then sends it back to the user through Send API.
Messages are logged into a database; those going to the bot (from the visitor) are logged when they're received, and those the bot responds with are logged through the 'echo' callback.
The problem with this is that the bot can reply to certain visitor commands (phrases) and tries to perform certain actions based on the input. I'm using Wit.ai for this, but due to the scope of the possible phrases and keywords, the default mode when someone sends a text message is to send it to Wit.ai for processing. However, if an admin user sends the visitor a message from the control panel, the visitor could want to respond to that message (instead of sending a bot command) and that response should not go to the Wit.ai for processing. And due to the sheer scope of possible variations of what can be said, coupled with the fact that they can actually use some of the keywords in the response as well, processing the intent with Wit.ai in that case is too uncertain.
I was wondering if there's a way to somehow identify/mark the source of the messages that the bot sends to the visitor, so when an echo callback comes, I can know if it's, say, from a regular bot routine or from a user-entered reply. Like some additional meta tags that could be sent with the message that would also get returned with the echo, but that doesn't pollute the message itself. Is something like that possible? Or is there a different way I can achieve the same result.
I don't wether that helps you, but Facebook just recently integrated a quite mysterious Tag feature for bots.
https://developers.facebook.com/docs/messenger-platform/send-api-reference/tags