Entity detection from sentences differ in training and testing sentences - nlp

Entity detection is working fine while training the phrases i.e the words of interest (entities) are highlighted when the sentence is being added as a training sample, but when the same sentence is tested , the entities are not selected.
If we look at example attached, the entity
price of property
was automatically selected during training but was empty/not detected during the test.

Please check if the value / entry "2 crores" is not duplicated in "price of property" entity.

In order for a reference value to be matched, it needs to be included as a synonym itself. When you first enter a reference value, it will be automatically added as a synonym. If for some reason this is removed, make sure to add the original value as a synonym so it can be matched.

Related

VBA Scenario Testing in a financial model

So I'm new to VBA and trying to automate my financial models. I've got this large complex stock valuation model on excel that reads either string bear, bull, or base from a single cell that has a dropdown that has the options of bear, bull, or base. The model will use different inputs depending on if the user chooses the strings bear, bull, or base, and the model will output different values of the stock respectively. In a separate sheet, I'm trying to display, in 3 separate cells, the output of the model (the value of the stock) in the bear, bull, and base case respectively. I could just make 3 models; a bear, bull, and base case, but I thought that might be too bulky, and I just want 1 variable model, instead of 3 fixed models. I'm wondering if VBA allows for a "scenario" type of ability in which it sets a specific cell to a hypothetical value and then takes the value of another cell, without actually modifying my model.
I think I understand what you want -- but it is still unclear. Perhaps provide an example?
One way I'd do it would be to have a fourth dynamic model. This extra model is based on the hypothetical scenarios (e.g. another column for Bear/Bull/Base for each stock -- we can call this extra column BBB2, and the original one with drop down menus for the user BBB1). The fourth model is based only on data from BBB2.
By default when the sheet is opened, BBB2 = BBB1. When a user changes BBB1, BBB2 gets updated. Then, I'd add a function to change everything in BBB2 to bull or base or bear. From there you could manually change a few stocks and see the results in the fourth model.

Which technique for training should I use for the following dataset?

I have a dataset which is a csv having 2 columns "Text", "Name".
"Text" column contains the news article.
"Name" column contains the extracted name from the corresponding text.
I have to train the model on this dataset, dataset contains 4000 plus unique news, where in, once your model is trained and validated, User should be able to pass any text and it should fetch the proper name.
What technique should I use and implement it. Please suggest.
Thanks in advance.
It sounds like you are looking to search for an item by keywords. In a basic case you could use a bag of words approach, in which you tokenise the words in the Text-field and index each document accordingly.
The relevance of each document can then be calculated given some measure (for instance cosine similarity).
You can find an example using the genesis-library here: https://radimrehurek.com/gensim/tut3.html
It is quite basic, note however that it does use LSI.

Two or more identical parameters in one sentence

I have a dialogflow intent which includes different parameters.Shown as below:
1.png
I set generic "number" for all number parameters such as "Years to Average Principal" and generic "percentage" for all percentage parameters.Like this:
2.png
When I say something like "Change vacancy rate to 56%" or "change Years to Average Principal to 5", it works well.But when I say a sentence which includes two or more parameters with same type, it sets the first value for all parameters.I used the "Change depreciation percentage to 30% and down payment to 45%" phrase and the result is following images:
3.png
4.png
As I have to use only one intent and one action,would you please help me what should I do to have separate values for different parameters?
I hope I could make it clear. Thanks in advance,
Good try using the "is list" setting for a parameter! Unfortunately, you only set it for one of the parameters, so it treated that as the list and then used the first matching value for the other parameters since both matched the rule for percentage. If one was for percentage and the other for currency, it might have worked better, but probably not.
Two (untested) thoughts about how to get this to work (and it is a good attempt to solve the problem of entering multiple values at a time):
You can try to see if setting all the other parameters as lists will work as well. But this still might have the problem of setting one parameter to a number and the other to a percentage.
Look into Developer Composite Entities. This will let you create a composite entity that contains both the name and value and treat it as a single Entity. Then you can specify, in your sample phrase, that you will accept a list of these Entities.

Null values for system entities

I tried an example in api.ai, with reference to the Beginner's Guide, I am attaching error response as a screen-shot. I was expecting to get the values of marked intents.
The problem is that, somehow, you have "tomorrow" associated with a city entity in the first example phrase (the purple color matches the #sys:geo-city-gb entity). Because of this, it is looking for a phrase "Can I know available slots for somewhere" where the somewhere should be a city name. But "tomorrow" is not the name of a city, so it doesn't include that.
Since neither parameter is required, but it has a pretty close match, it is able to match the intent - but none of the parameters fit, so they're all left empty.
You should be able to delete this and re-type "tomorrow" to change it back to a #sys.date entity. Or add additional phrases which match it against a date.

Excel Substitute given values and delete non given values

I have 3 columns named Old Model, New Model, & Obsolete Model. The New Model should be the output and is the New name from the Old Model table which is the input where as: AB
should be IJ, CD should be KL, and EFshould be MN. The Obsolete Model containing GH should not be included in the output of column New Model.
Old Model New Model Obsolete Model
AB,CD,EF,GH IJ,KL,MN GH
I can simply use the Substitute excel formula for this one which goes like this:
=SUBSTITUTE(SUBSTITUTE(SUBSTITUTE(A2,"AB","IJ"),"CD","KL"),"EF","MN")
The problem with this formula is that it will still show GH in the New Model column.
Can Anyone help me with this one? I have more than 1000 lines and within are different Old Model different New Model and different Obsolete Model
Is there a simple approch for this one without using VBA codes since the fili I am running is quite big and I'm affraid it might slow down the process.
Best Regards,
EDIT
Extract one old model from the cell like this:
=MID(A1,FIND("CD",A1),(len("CD")))
Result: AB,CD,EF,GH --> CD
Concatenate as many of those sequences as necessary to isolate all your old models.
=MID(A1,FIND("AB",A1),(len("AB")))&MID(A1,FIND("CD",A1),(len("CD")))&MID(A1,FIND("EF",A1),(len("EF")))
Then in a separate column, you an apply your original substitute formula to the result.
Add another substitute statement and substitute "GH" with nothing, "".
=SUBSTITUTE(SUBSTITUTE(SUBSTITUTE(SUBSTITUTE(A2,"AB","IJ"),"CD","KL"),"EF","MN"), "GH", "")
Then to isolate "GH" in the obsolete column, just replace all the other values with nothing.
=SUBSTITUTE(SUBSTITUTE(SUBSTITUTE(SUBSTITUTE(A2,"AB",""),"CD",""),"EF",""), "GH", "")
Make sure to add commas to the strings to be replaced if you want to get rid of those also, i.e. "AB,", "CD,", "EF,".

Resources