I created an ArangoSearch view over a collection and am using the SEARCH keyword with wildcards w/ LIKE to search a field with spaces, similarly to how MySQL would. The problem I am running in to is, I keep getting an empty set even though records with the Star Wars title definitely exist.
Note, searching for '%star%' works and returns results... as soon as I add a space and search for '%star wars%' the query returns empty set.
This is the query
FOR d IN imdb_norm
SEARCH ANALYZER(d.name LIKE "%Star Wars%", "text_en")
RETURN d.name
This is the structure, running arango version 3.7.2
The thing is since you're using text_en, it breaks up strings into individual words Since all spaces are considered as break characters, there is not a single term (a word stored in aragnosearch index) containing a whitespace.
If you don't need tokenization, please can consider indexing a word as it is (i.e. without case conversion, accent removal, etc) using identity analyzer, or check out norm analyzer instead.
https://www.arangodb.com/docs/stable/arangosearch-analyzers.html#analyzer-types
Related
I currently use Typesense to search in an HTML database. When I search for a term, I would like to retrieve N characters before and N characters after the term found in search.
For example, I search for "query" and this is the sentence that matches:
Let's repeat the query we made earlier with a group_by parameter
I would like to easy retrieve a fixed number of letters (or words) before and after the term to show it in a presumably small area where the search results is retrieved, without breaking any words.
For this particular example, I would be showing:
..repeat the query we made earlier..
Is there a feature like this in Typesense?
I have checked Typesense's documents, without any luck.
The feature you're referring to is called snippets/highlights and it's enabled by default. You can control how many words are returned on either side of the matched text using the highlight_affix_num_tokens search parameter, documented under the table here: https://typesense.org/docs/0.23.1/api/search.html#results-parameters
highlight_affix_num_tokens
The number of tokens that should surround the highlighted text on each side. This controls the length of the snippet.
I am trying to configure Azure Search to find some strings that have special characters, for example
ABC*DEF
When I look for a the full term using "ABC*DEF", it works perfectly.
The problem comes if I want to use a regex term:
When I use a partial term, like /(.*)ABC(.*)/, the result has no problem
When I use a partial term, like /(.*)DEF(.*)/, the result has no problem
But when I try to look for something like /(.*)C\*D(.*)/, the result is empty.
I am using a standard analyzer. I tried also the keyword analyzer but that way the regex search doesn't work at all.
Any suggestions?
You won't be able to create a regex expression that matches ABC*DEF using the standard analyzer.
If you run "ABC\*DEF" through the analyzer api using "standard" analyzer, you will see that ABC*DEF gets divided into 2 tokens at indexing time -> "ABC" and "DEF". Regex expression are not analyzed, however, they need to match a token that exist in the index.
Since ABC\*DEF does not exist in the index (only "ABC" and "DEF" exist), you won't be able to find it using the expression you are searching for.
Using the "keyword" analyzer will keep the whole field as a single token, so if the field "only" contained the expression ABC\*DEF, then the regex expression would work on it, however, if ABC\*DEF is part of a larger paragraph of text, then that's probably not what you want to use.
Your best bet is to create a custom analyzer that tokenizes your text in the way that preserves the special characters that are relevant to your use case.
If you're searching for special chars, why don't you discard normal chars?
[^\w]
I've made an azure search service and it's up and working. I would like for users to be able to search with 3 characters or more.
I have the following texts in different documents:
Paracet 200mg
Paracet 150mg
Kodein/paracetamol SA
When I search for 'par' I get no results. I have to type 5 characters (parac) and I get 1 & 2 as a result. I want this result for 'par' as well. Is this possible? I can't find anything in the documentation on setting the required number of characters for a search.
For the best performance, you could enable the "fast" prefix analyzer in your index, which will break down every token into a list of prefixes at indexing time. Here's some additional information on how to do that : https://azure.microsoft.com/en-us/blog/custom-analyzers-in-azure-search/
This would require you to re-index your data, so if you are creating a brand new index, this is an option.
If re-indexing is not an option, you can instead use the suffix operation '*' in your query. Here's more information on the suffix operator : https://learn.microsoft.com/en-us/rest/api/searchservice/Simple-query-syntax-in-Azure-Search?redirectedfrom=MSDN
I suspect searching using the suffix operator (or re-indexing while using fast prefix analyzer) will also work with the 3rd document you listed (Kodein/paracetamol SA). If it still does not work, it might be due to you using a tokenizer that does not split on the '/' character. The default analyzer should correctly split on '/', but if you are using a custom analyzer it's possible the whole "Kodein/paracetamol" expression get tokenized into a single term, which would explain why a search for parace* does not return the document, since the prefix of the document is "kode…".
I am trying to search for a term in Solr in the Title that contains only the string 1604-04. But the results come back with anything that contains 1604 or 04. What would the syntax be to force solr to search on the exact string of 1604-04?
You can also use Classic Tokenizer.The Classic Tokenizer preserves the same behavior as the Standard Tokenizer with the following exceptions:-
Words are split at hyphens, unless there is a number in the word, in which case the token is not split and
the numbers and hyphen(s) are preserved.
This means if someone searches for 1604-04 then this Tokenizer won't break search string into two tokens.
If you want exact matches only, use a string field or a text field with a KeywordTokenizer as the tokenizer. These will keep your tokens intact as one single entry, and won't break it up into multiple tokens.
The difference is that if you use a Textfield with a KeywordTokenizer, you can still apply other filters, such as a LowercaseFilter, while a string field will store anything verbatim without any further processing possible.
Your analyzer is splitting "1604-04" into two terms, "1604" and "04". You've received answer on how to change your analysis to stop doing that.
Changing your analysis my not be the best solution (can't be entirely sure based on what you've written). Using a phrase query would be the usual way to do this. You can use a phrase query by wrapping it in quotes:
field:"1604-04"
This will still analyze and split it into two terms, but it will look for those terms in sequence. So, that query would match "1604-04" and "1604 04", but not "1604 some other stuff 04".
I have stemming enabled in my Solr instance, I had assumed that in order to perform an exact word search without disabling stemming, it would be as simple as putting the word into quotes. This however does not appear to be the case?
Is there a simple way to achieve this?
There is a simple way, if what you're referring to is the "slop" (required similarity) as part of a fuzzy search (see the Lucene Query Syntax here).
For example, if I perform this search:
q=field_name:determine
I see results that contain "determine", "determining", "determined", etc.. If I then modify the query like so:
q=field_name:determine~1
I only see results that contain the word "determine". This is because I'm specifying a required similarity of 1, which means "exact match". I can specify this value anywhere from 0 to 1.
Another thing you can do is index the same text without stemming in one field, and with stemming in another. Boost the non-stemmed field & that should prefer exact versions of words to stemmed versions. Of course you could also write your own query parser that directs quoted phrases to the non-stemmed field only.