For example Intent cutoff has training phrase - ABC college cutoff
where ABC college is entity - college
cutoff is - collegedetail entity
Now another Intent exam has training phrase - ABC college exams
where ABC college is again entity - college
exams is again- collegedetail entity
Here entities are meant to be same but Intents are different. But Dialogflow is confusing the intents every time because of same entities.
So my question is should entities be Intent specific or there is some other way to handle this situation?
Parameters for the training phrases are meant to capture different values, all of which are valid for that Intent.
If the Exam intent has a training phrase "[ABC college][exams]" with each of those parts having an Entity, then it means that you're expecting different values for each part, the Entity reflects what those possible values should be, and you want to be able to find out what that specific value was.
It sounds like you're saying that the training phrase for the Exam Intent should be something more like "[ABC college] exams". So you have a parameter for the college being asked about, but you are expecting only "exams" to be a valid thing said for this Intent. You don't need to know exactly what that value was - you know its about exams because that's the purpose of all the training phrases for that Intent.
Related
I am trying to connect different entity types or entity properties. Consider this case:
I have a list of animals and I have a list of vegetables
Each animal eats a subset of vegetables - Cow eats veggieA,veggieB and Hen eats veggieB,veggieC
Once Cow is in context, the only vegetable entities I want to catch are veggieA and veggieB. Even if veggieC is mentioned, I do not have want to catch this entity for all my follow up intents
How can I achieve this?
Can I have an animals entity type with a list of animals like Cow,Hen etc and a vegetable entity type with a list of vegetables like veggieA,veggieB etc
Or like Each animal form a different entity type - Cow is a entity, Hen is a entity
How can I connect different entities like how I described earlier?
The key point to consider is that any value within an entity (custom or not) will be a valid match for a particular intent that has that entity as a parameter.
Meaning, if you have an animal entity and a vegetable entity, even if you train an intent Cow-intent only with cow as the animal this intent will be matched regardless of the animal you submit. As far as I know you cannot restrict the entity values that are valid within an intent.
For this reason, the easiest approach to ensure your behavior is met is to create separate entities for each animal (group of animals with the same veggies). Then create specific intents for each animal entity which follow-up intent/s will have a specific vegetable entity assigned as well. Meaning, a lot of particular entities instead of general ones. You would have entities: Cow-like,Hen-like,Cow-veggies,Hen-veggies,...
If that seems unfeasible for your particular use case you can also try to keep general entities and code some conditionals in the Webhook Fulfillment to handle your particular behavior, similar to what is done in here.
I'm using Dialogflow as the NLP engine behind a chatbot, and am trying to get it to recognize company names. In the following examples, it understands the intent well, but doesn't pick up the company name.
Create a company called Google
Make a new account called Johnson & Johnson
New company Nike
Does anyone have any advice on how I can get Dialogflow to start to recognize these entities? I'm wondering if there are features I don't know about, or maybe some sort of plugin/library I can utilize for this?
I'm afraid there's no Dialogflow system entity that can do this for you as of Oct, 2020. Your best bet is to add as many training phrases as possible and create a custom entity with #sys.any as the entity type. Annotate as many training phrases as possible and let Dialogflow do the rest. When it comes to identifying company names specifically, there are two types of company names:-
Common company names like "Google", "Facebook" which Dialogflow can recognize without much assistance, especially if your entity type is #sys.any.
Domain-specific company names like Overflow LLC or Stack and Overflow Associates. Here, the annotated training phrases play an important role and if you have an idea of the types of companies that would need to be understood, it would help annotating those phrases (Eg: LLC, Associates, Firm etc).
Also think about how you structure your question to ensure the user enters values as per your needs. Eg: Please type in/spell out the name of your company increases the chances that anything your user enters would just be the company name.
I'm trying to create a model in LUIS that allow me to detect if a brand (any brand) is mentioned in an utterance. I've tried different approaches but I'm struggling to get it working.
First I have an intent searchBrand with some examples utterances:
'Help me find info about Channel'
'I want to know more about Adidas'
...
What I want is that LUIS recognizes that a brand has been mentioned in the utterance (as an entity).
I believe I have these options:
Use a List Entity: impossible since I would have to fill the list
with every possible brand that exists and, moreover, the user would
have to write the brand exactly as it is, not allowing typos (e.g. ralf
lauren)
Use a ML Entity: I believe this could be the right approach. I've tried the following without success:
Create a ML Entity "brands"
Add a Structure with 1 component "brand"
Add to the component a Descriptor with a list of different brands as an example
Once I label the entities in the utterances, the model recognizes correctly the brands that I added to the Descriptor but it fails to recognize others brands or typos
Another option is a pattern entity. It fits somewhere between the two options you listed. You do need to train it with the patterns, and if the pattern is off at all it will not recognize the entity (and won't recognize the intent either unless you've separately trained it with utterances, which you should). However, it seems like the phrasings in your case would be consistent enough that you could define a few patterns for this, and as you train your bot from endpoint utterances you can add additional patterns as needed. Here is an example:
As I put this together I realized I'm ignoring [help me] and [find], essentially the pattern is "info about {brand}", which may or may not be appropriate depending on your other intents. If you say something different like "Tell me more about Adidas", the intent will be recognized (I trained it with your sample utterances), but the pattern, and therefore entity, will not.
Tutorial on using Patterns in LUIS
I got it working following this:
Create a ML Entity "brands"
Add to the entity a Descriptor with a list of different brands as an example. Remember to normalize the elements in the Descriptor
Add brands to the Descriptor
Label entities as "brands" inside utterances in intent "searchBrands"
Train & test the model
It is very important to normalize everything in LUIS. I had the brands inside the Descriptor capitalized and LUIS couldn't recognize new ones, once I normalized the brands LUIS started suggesting new ones and recognizing more when testing the model
When you create Intents and enter their sample utterances in LUIS, the parser will sometimes classify some words as being entities. This is a nice feature when it accurately identifies them, but sometimes it mislabels them.
For example, if you have an entity for statuses of a switch (on/off), constructed as a List with "true" and "false" being the values for which "on" and "off" are synonyms, respectively, then every time you use the words "on" or "off" (which have various meanings, uses and purposes) in an intent's sample utterances, they get labeled as that entity, often inaccurately.
The documentation (https://learn.microsoft.com/en-us/azure/cognitive-services/luis/luis-how-to-add-example-utterances) states that List type entities cannot be removed from utterances. Is there any way to avoid simple words that may be used as synonyms in entities from being matched as entities?
Thanks!
I think the only way to do it is to remove those simple words as synonyms (on, off, etc.) from the List entity synonyms (clicking x next to the synonym). Per the message when you create a List entity, they behave differently than other entity types and are direct matching:
Unlike other entity types, additional values for list entities aren't
discovered during training. This entity type is identified in
utterances by the direct matching of utterance text to the defined
values, rather than learning from context.
You could also use simple entities along with Phrase Lists to help boost the signal to those instances where on/off would be an entity that you would want to capture. Adequately supplied phrase lists to help identify those types of instances would be needed.
I am trying to develop software to get suitable attributes for entities names depending on entity type.
For example if I have entities such doctor, nurse, employee , customer, patient , lecturer , donor, user, developer, designer, driver, passenger and technician, they all will have attributes such as name, sex, date of birth , email address, home address and telephone number because all of them are people.
Second example word such as university, college, hospital, hotel and supermarket can share attributes such as name, address and telephone number because all of them could be organization.
Are there any Natural Language Processing tools and software could help me to achieve my goal.
I need to identify entity type as person or origination then I attached suitable attributes according to the entity type?
I have looked at Name Entity Recognition (NER) tool such as Stanford Name Entity recognizer which can extract Entity such as Person, Location, Organization, Money, time, Date and Percent But it was not really useful.
I can do it by building my own gazetteer however I do not prefer to go to this option unless I failed to do it automatically.
Any helps, suggestions and ideas will be appreciated.
If I understand correctly, you are mainly interested in knowing if a given word can be mapped to a general category of Human, Organization, etc.
You should use WordNet, which provides a complete hierarchy of the general English lexicon. Try it a bit in the user interface to get of feel of how it works.
WordNet encodes relations between words. One of these relation is hypernymy, a fancy word that means a relation of general-to-particular.
Some examples:
Vehicle is a hypernym of boat.
Vehicle is a hypernem of car.
Human is a hypernym of worker which is a hypernym of plumber.
Hyponymy is the inverse relation of hypernymy:
Boat is a hyponym of vehicle.
Car is a hyponym of vehicle.
Plumber is a hyponym of worker, itself a hyponym of human.
These relations are transitive, so in my last example plumber is also a hyponym of human. This gives you the solution to your problem: any word that has human as hypernym should be mapped to Human and have people attributes.
There are libraries to access WordNet from Java and Python, as well as from many other languages. Here is the documentation for using WordNet with the NLTK Python module.
A short example to determine if a word is hyponym of "human"
from nltk.corpus import wordnet as wn
human = wn.synset('person.n.01')
hyponyms_of_human = set(x for x in human.closure(lambda s:s.hyponyms())
fireman = wn.synsets('fireman')
salad = wn.synsets('salad')
print(any(x in hyponyms_of_human for x in fireman)) # outputs True
print(any(x in hyponyms_of_human for x in salad)) # outputs False