For example, lets say I want a command "center" that clicks the mouse in the center of the screen. Trivial example, but I'm more interested in the grammar aspects of it.
What if I only want to match "center"?
So if I pause, say "center", and then pause it is a match.
But if I say "I am in the center of the room" I do not get a match.
The following code seems to match the word "center" no matter what part of a phrase it is spoken in:
Choices center = new Choices( new string[] { "center" } );
SemanticResultKey centerKeys = new SemanticResultKey( "center", center );
GrammarBuilder centerGrammarBuilder = new GrammarBuilder();
centerGrammarBuilder.Append( centerKeys );
speechRecognitionEngine.UnloadAllGrammars();
speechRecognitionEngine.LoadGrammar(new Grammar(centerGrammarBuilder));
The speech engines usually do a decent job making sure they don't recognize an in-grammar word in the middle of a sentence. But not always, if you have found one of those edge cases where the engine recognizes terms mid sentence I can recommend two things to help.
Add a garbage rule to your grammar, then ignore any recognition event that contains the garbage rule. Usually, this is not recommended (but can work) since people rarely use the garbage rule tradeoffs are made during model building and performance suffers. Also you may notice that it works better in some language models rather than others. Again, this is a consequence of model building and tuning.
(inside a grxml it would look like this <ruleref special="GARBAGE"/>)
http://msdn.microsoft.com/en-us/library/system.speech.recognition.srgsgrammar.srgsruleref.garbage(v=VS.85).aspx
Check and tune the confidence of your resulting word. Even if it recognizes mid sentence you should get a much lower confidence score on that phrase. Unfortunately this tuning sometimes requires lots of audio to get correct.
Related
Let assume that I have a dataset of car accidents. Each accident has a textual description made using a set of cameras and other sensors.
Suppose now I have only the data of a single camera (e.g. the frontal) and I want to remove all the sentences of the description that are not related to it. I think a basic and easy solution could be to use a boolean retrieval system using a set of specific keywords to remove unwanted sentences, but I don't know neither if it is a good idea ner if it could work; could someone suggest me any idea? What kind of statistics might be useful to study this problem? Thanks
Regex could be one solution.
I created a regex matching the word "front", case insensitive, which searches for front and then get the whole sentences with one or more matches.
The results can be trimmed some from starting white spaces. (Can probably be removed as well with some fine tuning.)
The word you can swap out through some variable taking values from a list, if you need "front", "rear", "side", "right", "left" or other.
Regex Example https://regex101.com/r/ZHU0kr/5
I am working on an NLTK project, intended in principle to be like a standard thesaurus but (quasi-)continuous. To take one example, there are dozens of entries connected with books, including both religious classics and ledgers.
I tried fiddling with some terms, but I seemed to get just a smaller slice of the pie by doing that. (A "ledger" result contained "daybook" but the substances was a much smaller collection than one would find by reading a book.) The discussion of "synsets" in the documentation seem to imply both that you can find terms close to an existing terms, but the synsets are like islands, or see such to me.
What (if any) means are there to say something like "I want all words with a high match score above XYZ threshold" or "I want to match the n closest related terms." The documentation looks like this is possible, with a really nice way of calculating a proximity score between two words, but don't see how to adjust the threshold or alternately how to request the n closest matches.
What are my best bets here?
If you want to be able to compute distance between arbitrary pairs of words, WordNet is the wrong tool for the job: It is a network of particular terms, so either there is a path between two nodes or there is not. Look around for corpus-based measures instead.
A quick google gave this thread (not on SO) that could serve as a starting point.
In the nltk, I would start by taking a look at nltk.text.ContextIndex, which seems to be behind the nltk demo function nltk.Text.similar(). It won't calculate distances between pairs of words, but at least you'll have a rich network of contexts you can start from.
>>> contexts = nltk.text.ContextIndex(nltk.corpus.brown.words()[:100000])
>>> contexts.similar_words("fact")
['jury', 'announcement', 'Washington', 'addition', '1961', 'impression',
'news', 'belief', 'commissioners', 'Laos', 'return', '1959', '1960', '1956',
'result', 'University', 'opinion', 'work', 'course', 'hope']
I'll leave it to you to remove punctuation, stopwords etc. I haven't looked at the algorithms behind this, but you can always implement your own favorite algorithm if this doesn't do the job for you.
Sorry for the vague nature of this question, but I don't even know the proper NLP terminology to search constructively.
Basically what I'm attempting to do is to determine whether a word (usually an adjective or a noun) is "confirmed" by its context, or "negated." (Apologies for potential misuse of terminology)
The application is one step of a larger process that parses through a large body of user reviews, and uses keyword frequency to determine certain aspects of a product. However, certain keywords may either apply or not apply depending on the context.
As an example, when looking for a keyword "flimsy", two possible contexts are:
A) "The product felt flimsy in my hands and fell apart within days" --> keyword confirmed
B) "The build quality was very solid and not at all flimsy like its competitors" --> keyword negated
Note that this is different from the sentiment analysis of the phrase; the purpose is not to determine whether or not the opinion is positive or negative, but rather if a word is applicable in context or not.
Is there any standard methodology or technique to achieve this?
Suppose I have a set of phrases - about 10 000 - of average length - 7-20 words in which I want to find some given phrase. The phrase I am looking for could have some errors - for example miss one or two words, have some words misplaced, or some random words - for example my database contains "As I was riding my red bike, I saw Christine", and I want it to much "As I was riding my blue bike, saw Christine", or "I was riding my bike, I saw Christine and Marion". What could be some good approach to this problem? I know about Levenhstein's distance, and I also suppose that this problem may have no easy, good solution.
A good text search engine will provide capabilities such as you describe, fsh. A typical approach would be to create a query that matches if any of the words occurs and orders the results using a weight based on number of terms occurring in proximity to each other and weighted inversely to their probability of occurring, since uncommon words will be less likely to co-occur by chance. There's a whole theory of this sort of thing called information retrieval, but maybe you know about that. Furthermore you'd like to make sure that word-level fuzziness gets accounted for by normalizing case, punctuation and the like and applying some basic linguistic transformations (stemming), and in some cases introducing a dictionary of synonyms, especially when there is domain knowledge available to condition it.
If you're interested in messing around with this stuff, try an open-source search engine, this article by Vik gives a reasonable survey from the perspective of 2009, and this one by Middleton and Baeza-Yates gives a good detailed introduction to the topic.
I'm writing a bridge between the user and a search engine, not a search engine. Part of my value added will be inferring the intent of a query. The intent of a tracking number, stock symbol, or address is fairly obvious. If I can categorise a query, then I can decide if the user even needs to see search results. Of course, if I cannot, then they will see search results. I am currently designing this inference engine.
I'm writing a parser; it should take any given token and assign it a category. Here are some theoretical English examples:
"denver" is a USCITY and a PLACENAME
"aapl" is a NASDAQSYMBOL and a STOCKTICKERSYMBOL
"555 555 5555" is a USPHONENUMBER
I know that each of these cases will most likely require specific handling, however I'm not sure where to start.
Ideally I'd end up with something simple like:
queryCategory = magicCategoryFinder( query )
>print queryCategory
>"SOMECATEGORY or a list"
Natural language parsing is a complicated topic. One of the problems here is that determining what a word is depends on context and implied knowledge. Also, you're not so much interested in words as you are in groups of words. Consider, "New York City" is a place but its three words, two of which (new and city) have other meanings.
also you have to consider ambiguity, which is once again where context and implied knowledge comes in. For example, JAVA is (or was) a stock symbol for Sun Microsystems. It's also a programming language, a place and has meaning associated with coffee. How do you classify it? You'd need to know the context in which it was used.
And if you can solve that problem reliably you can make yourself very wealthy.
What's all this in aid of anyway?
To learn about "tagging" (the term of art for what you're trying to do), I suggest playing around with NLTK's tag module. More generally, NLTK, the Natural Language ToolKit, is an excellent toolkit (based on the Python programming language) for experimentation and learning in the field of Natural Language Processing (whether it's suitable for a given production application may be a different issue, esp. if said application requires very high speed processing on large volumes of data -- but, you have to walk before you can run!-).
You're bumping up against one of the hardest problems in computer science today... determining semantics from english context. This is the classic text mining problem and get into some very advanced topics. I thiink I would suggest thinking more about you're problem and see if you can a) go without categorization or b) perhaps utilize structural info such as document position or something to give you a hint (is either a city or placename or an undetermined) and maybe some lookup tables to help. ie stock symbols are pretty easy to create a pretty full lookup for. You might consider downloading CIA world factbook for a lookup of cities... etc.
As others have already pointed out, this is an exceptionally difficult task. The classic test is a pair of sentences:Time flies like an arrow.Fruit flies like a bananna.
In the first sentence, "flies" is a verb. In the second, it's part of a noun. In the first, "like" is an adverb, but in the second it's a verb. The context doesn't make this particularly easy to sort out either -- there's no obvious difference between "Time" and "Fruit" (both normally nouns). Likewise, "arrow" and "bananna" are both normally nouns.
It can be done -- but it really is decidedly non-trivial.
Although it might not help you much with disambiguation, you could use Cyc. It's a huge database of what things are that's intended to be used in AI applications (though I haven't heard any success stories).