I tried to create a custom named entity recognition model in Azure. I uploaded data, labled it, trained the model and it works pretty good. The only problem I have is, that the data type is not as I hoped. Thanks for your help
E.g. when I extract a birthdate and the entity is "tomorrow", the data type is not a date but a String. My question is, how to create a date out of the "tomorrow"?
Related
When I created A Parameter and I assigned it to te entity called #sys.unit-information-name,
I keep getting this warning: *The annotated text 'diabetes' in training phrase 'show me results on diabetes' does not correspond to entity type '#sys.unit-information-name'. And as a result my chatbot does not give me the right result, basically it will just keep asking the question under the Prompts under the Action and parameters.
If I remove this parameter it works. But I want to use the parameter & Entity
Any Help
Thanks
The problem you are facing is because you are explicitly creating a parameter and adding the system entity #sys.unit-information-name. This entity is used to refer to information about units as per system Entities documentation.
Since you are referring to a disease “diabetes” as per your expression “show me results on diabetes”, this entity cannot be used to map diabetes or any other disease also. There is no system Entities for Diseases, so it's better that we create a custom entity defining most of the disease names.
When giving the training Phrases if the word matches with values defined in entity, then entity mapping for that word will be done and the response will be generated.
I'm using Spacy and need some help to train our model with custom entities given in tabular format in a word/pdf document.
I'm able to train it with a custom entity based on an example of ANIMAL and it's working fine. In this case, we are providing the start and the end index of the aforementioned custom entity in a given text.
("Horses are too tall and they pretend to care about your feelings", {
'entities': [(0, 6, 'ANIMAL')]
}),
My question comes in case of Tabular format:
How can I give indexes like ANIMAL example?
Can anyone please guide and assist?
After a lots of research and article, I found a way to pass it through.
Convert this table as text.
As you convert this as text. this will add lots of white spaces etc.
Replace them with spaces.
This will convert you table as paragraph.
Now you can give indexes as sentences, and train your model.
Further, you can use dependency parser algorithm to find correct values linked with head ( in case, a values belongs to multiple key)
You can also simply use pd.read_html([[pass your html here]]) and this will return list of dataframes which you can use.
Thanks.
I want to fetch the Hollywood movies data. I have referred many APIs like IMDB, omdb, etc but all these APIs are taking a query string as a required parameter.
But i want some dummy data of movies without passing query string as a parameter with all details of movies like title, poster_image etc.
So please tell me something so that I can move further on this.
This is not a direct answer, however, have you tried googling "imdb dataset"
Here is what i have found, you need to register on their site and download CSV. Which you can later import to your DB.
IMDB Dataset - Kaggle
I want to create an entity that has any value except the values that are defined in another entity.
For example, i have an entity that contains all the possible products categories that i use in the bot, and if the user type a value that is not in that entity i want to react in some way.
It's like a fallback but only triggered when that condition is met.
Any suggestion?
Entity extraction is based on some definite value that can be identified and separated. There should be some basic features defined for the agent to train on. Based on these trained features, the agent will look for an entity and extract it from the user's response.
If you have already defined an entity to look for, it will be extracted by the Dialogflow based on the training data. If there is nothing defined it will not be identified as an entity as the agent will be not sure what to look for.
So, what you can do is,
Make the entity (already defined) as not required. Uncheck the "required" checkbox in the Dialogflow.
Add the "#sys.any" in the Entity you defined and make it a composite Entity with the combination of your Entity and "#sys.any" something in the line of
Train your agent to look for this new Entity with your Basic Entity data and Anything else data.
Collect this in the webhook.
OR
You when you want to collect anything else, you can collect user utterance from the agent object and parse the data using Regex pattern of your choice.
Referring to the batch testing document from Microsoft, I have imported the LUIS app provided from this document and import this this JSON batch file to LUIS batch testing panel and try to run the test.
The testing result of 'Intents' is as expected, however, for the 'Entities', it is expected some of the entities will pass but at the end all entities are failed:
Entities Fail Screenshot
Is it a bug in LUIS or anything going wrong?
Updates: The issue above is fixed but some batch testing function are still not working
After the fix have been released, I have tried it again. I found that it works fine with no entity, simple entity and hierarchical entity. However, for List Entity and Regex Entity still not working on Microsoft sample. Entity list and entity list in batch testing in Microsoft sample are not match. Besides, in my own Luis project, the list entity is showing in the batch testing entity list but the result is strange (caption in this screen cap).
By the way, if you look at the tutorial, it is stated that:
"Only the machine-learned entities are used: simple, hierarchical (parent-only), and composite entities. Do not add non-machine-learned entities because they are always found either through regular expressions, or explicit text matches"
And List Entities are not machine-learned entities.