Is there a way to find a list of associated words to a Named Entity?
For instance : let the Named Entity be FIFA. Now FIFA is a Football Organization and hence related to the term football and all the terms related to football like jersey, footballers, goal, goalkeeper, halftime, penalty, freekick et cetera.
Is there a way we could do that? I have tried Babelnet but didn't get the desired results or anything even close to what I want.
Related
I am working on a project using NLP and I am trying to do something like this :
Does anyone know what this kind of tasks is called ?
This is a semantic lookup, where you look at specific terms in the text and find superordinates for them:
Chlorpromazine is a medicine or a drug; a diagnosis is a type of procedure, and a patient is a living being). You could do this with an ontology or a thesaurus; some dictionaries also have information about superordinates, often in the definition ("a horse is a mammal with four legs").
This is not named entity recognition. In NER you identify something by its name, and return a specific identifier. So for Chlorpromazine you would not say that it was a medicine/drug, but you would get a serial number or reference for it which is distinct from any other drug.
I'm facing this issue with geo places. For example.
USA, United states, United states of America, US.
Luis is able to detect them as built in entity of geographyV2
but I want NLP to pass all the similar terms of the user input. Ex user said US we send back geographyV2 entity and the similar synonyms of it , can luis do that by any chance? I'm desperate for this
It does not appear this is possible using the built-in geographyv2 entity. The entity just returns the value that was entered and the category is belongs to (country/region, city, point of interest, etc.). If you use the verbose response it will also give you additional information such as the starting index and length. But I can't find any option to have it return all related values/alternate names.
GeographyV2 prebuilt entity for LUIS app
Even if you create a custom list entity in LUIS with a normalized value and synonyms, I don't think you can get the full list of synonyms for that entity, only the normalized value. So if you need to see a list of alternate spellings, you're going to need some other function or API. I did a brief search and there's not much out there. The best I found was REST Countries, which you can pass a partial or alternate name in, and it will give you a bunch of information in the response. You may have to do some legwork, though, because it will give multiple matches (e.g. a search for United States returns results for United States of America and United States Minor Outlaying Islands). This is as close as I can figure out on how to solve your issue.
Hope all is well during these challenging times!
I would like to put it out there with regards to the best practice on Extending Dialogflow Entities and tackling non US/UK english locale input.
My challenge was to get accurate location values for Suburbs and Cities (parameters) in Australia. My team has tested many different ways of doing so with just using the standard system entities but non was able to capture the above accurately. It seems most of the location based entity types are geared towards US and UK but not AU (and perhaps other countries non US/UK).
The team had to implement a custom entity that has the list of suburbs and cities for australia. A roadblock that we came across shows that you can only grab 10,000 max entities (else Dialogflow and AoG won't sync/update). Our procured list has more than that for suburbs and hence we have to do a fair bit of work trimming it down which we did.
Question is, are we doing it the right way -> custom entity with csv list imported? For instance should we use the training path instead and stick to Sys.Geo-City and Sys.location?
Yes, if your set of suburbs and cities can be organized in this way, uploading a CSV would provide better accuracy for your use case. If you are over 10K values for a given entity type, you can split it into several entities and allow an intent to match either entity.
Project Background:
I'm quite new to NLP so please forgive me if my problem seems unreasonably complicated. I am trying to extract some features, like company names, some money value and names of individuals from a public company listing document, which large body of text (300+ pages).
Text parsed into the program looks something like this:
"In this motion, Company A Holdings (The "Company"), was sponsored by Company B Limited. John Doe, the chairman of the company, has approved of this activity"
The expected outcome looks something like this:
The Company: Company A Holdings
Sponsor: Company B Limited
Chairman: John Doe
Since all documents came in PDF form, I have parsed them in as text. Performed some NER with Spacy with the document I have, and based on the looks of the NER result, it had successfully recognised all the entities that I needed. (I.E. It recognised Company A Holdings, Company B Limited and John Doe)
How should I approach the said goal? I don't have a massive amount of files to train the model (currently around 30 ish documents), a general direction or example of modules on how to tackle the problem would be highly appreciated.
Thank you all in advance!
I am using "list" entity. However, I do not achieve my expected result.
Here is what I have for LUIS intent:
getAnimal
I want to get a cat**[animal]**.
Here is what I have with LUIS entities:
List Entities [animal]
cat: russian blue, persian cat, british shorthair
dog: bulldog, german shepard, beagle
rabbit: holland lop, american fuzzy lop, florida white
Here is what I have with LUIS Phrase lists:
Phrase lists [animal_phrase]
cat, russian blue, persian cat, british shorthair, dog, bulldog, german shepard, beagle, etc
Desired:
When user enters "I want to get a beagle." It will be match with "getAnimal" intent.
Actual:
When user enters "I want to get a beagle." It will be match with "None" intent.
Please help. Your help will be appreciated.
So using a phrase list is a good way to start, however you need to make sure you provide enough data for LUIS to be able to learn the intents as well as the entities separate from the phrase list. Most likely you need to add more utterances.
Additionally, if your end goal is to have LUIS recognize the getAnimal intent, I would do away with the list entity, and instead use a simple entity to take advantage of LUIS's machine learning, and do so in combination with a phrase list to boost the signal to what an animal may look like.
As the documentation on phrase lists states,
Features help LUIS recognize both intents and entities, but features
are not intents or entities themselves. Instead, features might
provide examples of related terms.
--Features, in machine learning, being a distinguishing trait or attribute of data that your system observes, and what you add to a group/class when using a phrase list
Start by
1. Creating a simple entity called Animal
2. Add more utterances to your getAnimal intent.
Following best practices outlined here, you should include at least 15 utterances per intent. Make sure to include plenty of examples of the Animal entity.
3. Be mindful to include variation in your utterances that are valuable to LUIS's learning (different word order, tense, grammatical correctness, length of utterance and entities themselves). Highly recommend reading this StackOverflow answer I wrote on how to build your app properly get accurate entity detection if you want more elaboration.
above blue highlighted words are tokens labeled to the simple Animal entity
3. Use a phrase list.
Be sure to include values that are not just 1 word long, but 2, 3, and 4 words long in length, as different animal names may possibly be that long in length (e.g. cavalier king charles spaniel, irish setter, english springer spaniel, etc.) I also included 40 animal breed names. Don't be shy about adding Related Values suggested to you into your phrase list.
After training your app to update it with your changes, prosper!
Below "I want a beagle" reaches the proper intent. LUIS will even be able to detect animals that were not entered in the app in entity extraction.