I know that i can make a none intent to cover some of these, however we cannot just create every nonsense question a person could ask.
Or even if someone types in a 50 word statement. The bigger problem is that if we get a query to LUIS, it is assigning it an intent that is not correct, without even having identified any entities either.
What to do?
To handle these cases, it would be better to add more labeled utterances to your other intents and occasionally add the stray utterances to the None intent. When the model is better for predicting your non-None intents, the better predicting of None intents also accompany this (LUIS attempts to match to an intent rather than cutting intents out).
If intents are triggering without any entities being recognized (and thus you believe the wrong intent has been triggered), this should be handled at an application level, where you would then disambiguate the intents back to your users. If you've set the verbose flag to true, then you could take the top three scoring intents and present those back as options to your user. Then you can move back into the proper dialog.
After you've moved into the intent/dialog they meant to access, you can conduct a programmatic API call to add that utterance to the intent. Individually adding labeled utterances can be problematic (the programmatic API key has a limit of 100,000 transactions per month, and a rate of 10 transactions per second), so you can instead aggregate the utterances and conduct batch labeling. An additional bit of info; there is a limit of 100 labeled utterance per batch upload.
Adding to the Steven's answer - in the intent window, you have the Suggested Utternaces tab - this is also a hint for the algorithm, kind of reinforced learning approach.
Related
Hi so I have a problem.
In Dialogflow, when I get a response to end the chat, I would like to ask the user for ratings.
so I've created 2 intents, "endchat" and "endchat2."
They both have the same training phrases, but it appears only endchat2 is being used (the most recently created intent)
How do I ensure that the chatbot randomly chooses an intent after a given response, instead of only using one intent? They have the same training phrases.
An alternate idea is in the attachments. The problem lies that I want the custom payload to only to appear after one of the text responses, (that being text response #1,) but not appear, if the chatbot decides to use text response #2. This is the reason I decide to make two separate intents, but it looks like that's not helping out because the bot is only using one intent.
Remember, Intents represent what the user says and does and not how you respond to that. So there is no way to "randomly choose an Intent" to use to respond.
What you can do, however, is setup a webhook for that Intent and determine how you wish to respond to what the user says. In some cases, you can thank them and end the conversation, while in others you can thank them, ask them the followup question, and set a Context so you can expect their reply.
Having the same / similar training phrase in multiple intents is an anti-pattern of bot design. Ultimately this confuses the bot and it leads to undefined behavior.
This should also trigger an warning in "Validation" with something like "Multiple intents share training phrases which are too similar:..." on the intents.
I have some question about some "best practice" for certain issues that we are facing using LUIS, QnA Maker, in particular for the Dispatcher:
1) Is there any best practice in case we have more that 15k utterances in the Dispatcher? That's looks like a limitation of the LUIS apps but the scalability of the model in the long run will be questionable.
2) Bing Spell Check for LUIS changes names and surnames for example, how to avoid this? I guess that Bing Spell Check is necessary when we are talking about ChatBots, since the typo are always behind the door, but using it for names is dangerous.
3) Cross validation is not supported out of the box, you would have split your data to folds with custom code (not difficult), use the command line to train and publish your model on your k-1/k folds, then send the k-fold utterances to the API one-by-one. Batch upload is only supported through the UI https://cognitive.uservoice.com/forums/551524-language-understanding-luis/suggestions/20082157-add-api-to-batch-test-model and is limited to a test set of 1,000 utterances. If we use the one-by-one approach, we pay $1,50 per 1k transactions https://azure.microsoft.com/de-de/pricing/details/cognitive-services/language-understanding-intelligent-services/ and this means to get cross-validation metrics for the 5 folds for example, we could be paying about 20$ for a single experiment with our current data, more if we add more data.
4) Model is a black box, which doesn't give us the ability to use custom features if needed.
I will try to address your concerns in the best possible way I can as follows:
1) As per the LUIS documentation,
Hence, you cannot exceed the limit. In case of Dispatch apps,if the total utterance exceeds 15k, then dispatch will down sample the utterances to keep it under 15k. There is an optional parameter(--doAutoActiveLearning) for CLI to do auto active learning which will down sample intelligently (remove non relevant utterances).
--doAutoActiveLearning: (optional) Default to false. LUIS limit on training-set size is 15000. When a LUIS app has much more utterances for training, Dispatch's auto active learning process can intelligently down sample the utterances.
2) Bing Spell Check helps users to correct misspelled words in utterances before LUIS predicts the score and entities of the utterance. However, if you want to avoid using Bing Spell Check API service, then you will need to add the correct and incorrect spelling which can be done in two ways:
Label example utterances that have the all the different spellings so that LUIS can learn proper spelling as well as typos. This option requires more labeling effort than using a spell checker.
Create a phrase list with all variations of the word. With this solution, you do not need to label the word variations in the example utterances.
3) As per the current documentation, a maximum of 1000 utterances are allowed per test. The data set is a JSON-formatted file containing a maximum of 1,000 labeled non-duplicate utterances. You can test up to 10 data sets in an app. If you need to test more, delete a data set and then add a new one. I would suggest you to report it as a feature request in the feedback forum.
Hope this helps.
I am trying to make a search algorithm with dialogflow that could take any combination of: first name, address, phone number, zip code or city as input to a search algorithm. The user does not need all of them, but we will refine our search with each additional answer until we only have one result. Basically we are trying to identify which customer we are talking to.
How should this type of intent (or set of intents) be structured? We have tried one intent with multiple parameters, but we do not need all of them to be required. We have also written a JavaScript function for fulfillment but how can we communicate back to dialogflow as to whether we need more information?
Thank you very much for your help.
Slot filling is designed for this purpose.
Hope that helps.
Please post more code/details to help answers be more specific.
First, keep in mind that Intents reflect what the user is saying, and not typically what you're replying with or what other information you need. Slot filling sometimes bends this rule, but only if you have required slots.
Since you don't - you need a different approach.
This can be done with a single intent, although you may find that multiple intents make it easier in some ways. The approach is broadly the same:
When you ask the question, make sure you set an Outgoing Context with a relatively short lifespan (2-3 is good) to indicate you are collecting user info.
Create an Intent (or Intents) that have sample phrases that capture the information you need.
Some of these will have obvious entity types (phone number and zip code) while others will be more difficult (First name has a system entity type, but it doesn't include all possible first names).
You will need to create sample phrases that collect the parameters by themselves, along with phrases that make sense. You're the best judge of this, and you should probably write some sample conversations before you write the phrases.
In your fulfillment, you'll figure out if you have enough information.
If you do, you can reply and clear the Context that was set. (Clearing it is important so Dialogflow doesn't match the information collecting Intent again.)
If you do not, you can add the information you have as parameters to the Context so you can save it for later processing, make sure you reset the Context lifespan (so it doesn't expire), and prompt the user for additional information. Again, having a conversation mocked out ahead of time will help here.
I have created a pizza bot in dialogflow. The scenario is like..
Bot says: Hi What do you want.
User says : I want pizza.
If the user says I want watermelon or I love pizza then dialogflow should respond with error message and ask the same question again. After getting a valid response from the user the bot should prompt the second like
Bot says: What kind of pizza do you want.
User says: I want mushroom(any) pizza.
If the user gives some garbage data like I want icecream or I want good pizza then again bot has to respond with an error and should ask the same question. I have trained the bot with the intents but the problem is validating the user input.
How can I make it possible in dialogflow?
A glimpse of training data & output
If you have already created different training phrases, then invalid phrases will typically trigger the Fallback Intent. If you're just using #sys.any as a parameter type, then it will fill it with anything, so you should define more narrow Entity Types.
In the example Intent you provided, you have a number of training phrases, but Dialogflow uses these training phrases as guidance, not as absolute strings that must be matched. From what you've trained it, it appears that phrases such as "I want .+ pizza" should be matched, so the NLU model might read it that way.
To narrow exactly what you're looking for, you might wish to create an Entity Type to handle pizza flavors. This will help narrow how the NLU model will interpret what the user will say. It also makes it easier for you to understand what type of pizza they're asking for, since you can examine just the parameters, and not have to parse the entire string again.
How you handle this in the Fallback Intent depends on how the rest of your system works. The most straightforward is to use your Fulfillment webhook to determine what state of your questioning you're in and either repeat the question or provide additional guidance.
Remember, also, that the conversation could go something like this:
Bot says: Hi What do you want.
User says : I want a mushroom pizza.
They've skipped over one of your questions (which wasn't necessary in this case). This is normal for a conversational UI, so you need to be prepared for it.
The type of pizzas (eg mushroom, chicken etc) should be a custom entity.
Then at your intent you should define the training phrases as you have but make sure that the entity is marked and that you also add a template for the user's response:
There are 3 main things you need to note here:
The entities are marked
A template is used. To create a template click on the quote symbol in the training phrases as the image below shows. Make sure that again your entity is used here
Make your pizza type a required parameter. That way it won't advance to the next question unless a valid answer is provided.
One final advice is to put some more effort in designing the interaction and the responses. Greeting your users with "what do you want" isn't the best experience. Also, with your approach you're trying to force them into one specific path but this is not how a conversational app should be. You can find more about this here.
A better experience would be to greet the users, explain what they can do with your app and let them know about their options. Example:
- Hi, welcome to the Pizza App! I'm here to help you find the perfect pizza for you [note: here you need to add any other actions your bot can perform, like track an order for instance]! Our most popular pizzas are mushroom, chicken and margarita? Do you know what you want already or do you need help?
Suppose i want to make a pizza ordering DialogFlow agent. To order a pizza we need 3 things: size, type and toppings.
If we want to go with follow-up intents approach rather than using entities then there will be so many combinations in which user might provide the information.
1: i want a pizza -> no info
2: i want small pizza -> size
3: i want small cheese pizza -> size and type
4: i want small cheese pizza with olives -> size, type and toppings
5: i want small pizza with olives -> size and toppings
...
and so on
How to solve this problem?
There will be so many combinations if we have more entities (2^n combinations)
Note 1: cannot take entities and slotfilling option as there are so many problems if we go down that road, like re-prompts loop, validation etc.
Is there any better solution?
Note 2: If we use entities, mark them required, and set prompts then many times if it does not get desired input from user it get stuck in re-prompt loop, i.e it keeps asking user same (or random) prompt for same entity. In my use case, it is bad for user experience. If we use follow-up intents instead, then we can set fallback intents for all those intents which solved this problem. (please note that this is just example of the use case)
This is another example of why I used follow-up intents, it solved my date capturing problem as well. I took #sys.date.recent and set a fallback intent to capture inputs like last week, last month etc, this was not possible using slots.
First, remember that Intents should reflect what the user says, not necessarily what you are doing.
On the surface, it isn't clear why slot filling (either with fulfillment or using built-in prompts) won't meet your needs. Since you've indicated that all three bits of information are necessary (size, type, and toppings), you can mark them as such in your phrases and Dialogflow will prompt for the missing information until it gets everything.
You almost certainly do not want to use Followup Intents. These are good when you always have a specific response that you send that will always have a very narrow set of replies from the user, but are very poor if the response from your action will prompt the user to reply in many different ways.
Instead, I would use a related concept: Contexts. (At least if you're not going to use slot filling.) When you ask the question of what they want, set a Context so you know they are. Then have one or more Intents that have this as an Input Context that accept the various things the user might say. Your webhook should see if you have the information you need and, if not, prompt them what else you're looking for. At the end, prompt for a confirmation, but they may say something that adjusts the order.