To decode following Solr query - search
In our hybris schema, we are using LowerCaseFilterFactory. Also, the name/description is of type "text". Hence, solr treats 'ConsTRUCTION' and 'construction' the same way.
However, if I search for (two differnt keywords combined) toysChildren, then I get many results. However, toyschildren has no results.
So, wish to decode following code to understand what's letting it do that.
solrQuery-toysChildren:
q=_query_:"\{\!multiMaxScore\+tie%3D0.0\}\(\(code_text\:toysChildren\^90.0\)\
+OR\+\(keywords_text_en_mv\:toysChildren\^100.0\)\
+OR\+\(name_text_en\:toysChildren\)\)\
+OR\+\(\(keywords_text_en_mv\:toysChildren\~\^10.0\)\)\
+OR\+\(\(keywords_text_en_mv\:toysChildren\*\^50.0\)\
+OR\+\(name_text_en\:toysChildren\*\^45.0\)\)\
+OR\+\(\(keywords_text_en_mv\:\"toysChildren\"\^100.0\)\
+OR\+\(name_text_en\:\"toysChildren\"\~0.0\^90.0\)\)"
&sort=score+desc&start=0&rows=100&facet.field=gender_string_mv
&facet.field=price_gbp_string
&facet.field=categoryPath_string_mv
&facet.field=allCategories_string_mv
&facet.field=excludeFromGiftFinder_boolean
&facet.field=productVisible_boolean
&facet.field=category_string_mv
&facet.field=brand_string_mv
&facet.field={!ex%3Dfk8}productType_string
&facet.field=age_string_mv
&facet=true
&fq=productVisible_boolean:true
&fq={!tag%3Dfk8}productType_string:(BUNDLE+OR+REGULAR+OR+ESD)
&fq=(catalogId:"coreProductCatalog"+AND+catalogVersion:"Online")
&facet.sort=count
&facet.mincount=1
&facet.limit=50
&spellcheck=true
&spellcheck.q=toysChildren&spellcheck.dictionary=en
&spellcheck.collate=true
Note: Above is the solrQuery formed up in DefaultFacetSearchStrategy of type "SolrQuery".
Query response is:
{responseHeader={status=0,QTime=18,params={facet.field=[gender_string_mv, price_gbp_string, categoryPath_string_mv, allCategories_string_mv, excludeFromGiftFinder_boolean, productVisible_boolean, category_string_mv, brand_string_mv, {!ex=fk8}productType_string, age_string_mv],spellcheck.dictionary=en,start=0,sort=score desc,fq=[productVisible_boolean:true, {!tag=fk8}productType_string:(BUNDLE OR REGULAR OR ESD), (catalogId:"coreProductCatalog" AND catalogVersion:"Online")],rows=100,version=2,q=_query_:"\{\!multiMaxScore\ tie=0.0\}\(\(code_text\:toysChildren\^90.0\)\ OR\ \(keywords_text_en_mv\:toysChildren\^100.0\)\ OR\ \(name_text_en\:toysChildren\)\)\ OR\ \(\(keywords_text_en_mv\:toysChildren\~\^10.0\)\)\ OR\ \(\(keywords_text_en_mv\:toysChildren\*\^50.0\)\ OR\ \(name_text_en\:toysChildren\*\^45.0\)\)\ OR\ \(\(keywords_text_en_mv\:\"toysChildren\"\^100.0\)\ OR\ \(name_text_en\:\"toysChildren\"\~0.0\^90.0\)\)",facet.limit=50,spellcheck.q=toysChildren,spellcheck=true,facet.mincount=1,facet=true,wt=javabin,facet.sort=count,spellcheck.collate=true}},response={numFound=3,start=0,docs=[SolrDocument{indexOperationId_long=79, id=coreProductCatalog/Online/100310, pk=8796107702273, catalogId=coreProductCatalog, catalogVersion=Online, allCategoryCodes_string=/SM06010425/SM060104/SM0601, price_gbp_string=£0 - £19.99, allCategories_string_mv=[SM06010425, SM0601, SM060104], category_string_mv=[SM06010425, SM0601, SM060104], rating_double=5.0, totalReviews_int=1, productType_string=REGULAR, excludeFromGiftFinder_boolean=true, pictureJson_string={"240":"https://image.smythstoys.com/picture/desktop/100310.jpg","220":"https://image.smythstoys.com/picture/tablet/100310.jpg","180":"https://image.smythstoys.com/picture/mobile/100310.jpg"}, gender_string_mv=[Female], autosuggest_en=[Sylvanian Families, Toys, Fashion & Dolls, Sylvanian Children's Bedroom Furniture], spellcheck_en=[Sylvanian Families, Toys, Fashion & Dolls, With 2 beech-style beds which can be stacked on top of each to make, Sylvanian Children's Bedroom Furniture], categoryName_text_en_mv=[Sylvanian Families, Toys, Fashion & Dolls], productVisible_boolean=true, url_en_string=/toys/fashion-and-dolls/sylvanian-families/sylvanian-children-s-bedroom-furniture/p/100310, pictureMap_string={min-width:1200=https://image.smythstoys.com/picture/desktop/100310.jpg, min-width:768=https://image.smythstoys.com/picture/tablet/100310.jpg, max-width:768=https://image.smythstoys.com/picture/mobile/100310.jpg}, priceValue_gbp_double=11.99, categoryNamePath_string_mv=[Toys, Toys > Fashion & Dolls, Toys > Fashion & Dolls > Sylvanian Families], categoryMetaTitle_string=SM06010425_Sylvanian Families: Awesome deals only at Smyths Toys UK, categoryMetaDescription_string=Sylvanian Families! Shop for an excellent range. Watch out for great offers at Smyths Toys UK, code_text=100310, description_text_en=With 2 beech-style beds which can be stacked on top of each to make, name_text_en=Sylvanian Children's Bedroom Furniture, name_sortable_en_sortabletext=Sylvanian Children's Bedroom Furniture, brand_string_mv=[Sylvanian], age_string_mv=[6 - 8 Years, 3 - 5 Years], categoryPath_string_mv=[/SM0601/SM060104, /SM0601, /SM0601/SM060104/SM06010425], customCategoryPath_string_mv=[/curl/toys/c/SM0601, /curl/toys/c/SM0601/curl/toys/fashion-and-dolls/c/SM060104, /curl/toys/c/SM0601/curl/toys/fashion-and-dolls/c/SM060104/curl/toys/fashion-and-dolls/sylvanian-families/c/SM06010425], ukBestsellerRating_en_int=999999, ukBestsellerRating_sortable_en_int=999999, pictureUrl_string=https://image.smythstoys.com/picture/desktop/100310.jpg, _version_=1577946952144781312}, SolrDocument{indexOperationId_long=79, id=coreProductCatalog/Online/100471, pk=8796108128257, catalogId=coreProductCatalog, catalogVersion=Online, allCategoryCodes_string=/SM06010326/SM060103/SM0601, price_gbp_string=£0 - £19.99, allCategories_string_mv=[SM06010326, SM0601, SM060103], category_string_mv=[SM06010326, SM0601, SM060103], rating_double=4.3, totalReviews_int=4, productType_string=REGULAR, excludeFromGiftFinder_boolean=false, pictureJson_string={"240":"https://image.smythstoys.com/picture/desktop/100471.jpg","220":"https://image.smythstoys.com/picture/tablet/100471.jpg","180":"https://image.smythstoys.com/picture/mobile/100471.jpg"}, gender_string_mv=[Male], autosuggest_en=[Vtech Infant, Toys, Pre-School & Electronic Learning, Toy Story Mr. Potato Head], spellcheck_en=[Vtech Infant, Toys, Pre-School & Electronic Learning, Includes lots of accessories and a special compartment for, Toy Story Mr. Potato Head], categoryName_text_en_mv=[Vtech Infant, Toys, Pre-School & Electronic Learning], productVisible_boolean=true, url_en_string=/toys/pre-school-and-electronic-learning/vtech-infant/toy-story-mr-potato-head/p/100471, pictureMap_string={min-width:1200=https://image.smythstoys.com/picture/desktop/100471.jpg, min-width:768=https://image.smythstoys.com/picture/tablet/100471.jpg, max-width:768=https://image.smythstoys.com/picture/mobile/100471.jpg}, priceValue_gbp_double=9.99, categoryNamePath_string_mv=[Toys, Toys > Pre-School & Electronic Learning, Toys > Pre-School & Electronic Learning > Vtech Infant], categoryMetaTitle_string=SM06010326_Vtech Infant: Awesome deals only at Smyths Toys UK, categoryMetaDescription_string=Vtech Infant! Shop for an excellent range. Watch out for great offers at Smyths Toys UK, code_text=100471, description_text_en=Includes lots of accessories and a special compartment for, name_text_en=Toy Story Mr. Potato Head, name_sortable_en_sortabletext=Toy Story Mr. Potato Head, brand_string_mv=[Toy Story], age_string_mv=[9 - 11 Years, 6 - 8 Years, 3 - 5 Years], categoryPath_string_mv=[/SM0601/SM060103/SM06010326, /SM0601/SM060103, /SM0601], customCategoryPath_string_mv=[/curl/toys/c/SM0601, /curl/toys/c/SM0601/curl/toys/pre-school-and-electronic-learning/c/SM060103, /curl/toys/c/SM0601/curl/toys/pre-school-and-electronic-learning/c/SM060103/curl/toys/pre-school-and-electronic-learning/vtech-infant/c/SM06010326], ukBestsellerRating_en_int=999999, ukBestsellerRating_sortable_en_int=999999, pictureUrl_string=https://image.smythstoys.com/picture/desktop/100471.jpg, _version_=1577946952157364224}, SolrDocument{indexOperationId_long=79, id=coreProductCatalog/Online/100838, pk=8796111962113, catalogId=coreProductCatalog, catalogVersion=Online, allCategoryCodes_string=/SM060307/SM0603, price_gbp_string=£0 - £19.99, allCategories_string_mv=[SM060307, SM0603], category_string_mv=[SM060307, SM0603], rating_double=0.0, productType_string=REGULAR, excludeFromGiftFinder_boolean=false, pictureJson_string={"240":"https://image.smythstoys.com/picture/desktop/100838.jpg","220":"https://image.smythstoys.com/picture/tablet/100838.jpg","180":"https://image.smythstoys.com/picture/mobile/100838.jpg"}, gender_string_mv=[Female], autosuggest_en=[Sports Equipment, Outdoor, 8oz Childrens Boxing Gloves], spellcheck_en=[Sports Equipment, Outdoor, 8oz childrens boxing gloves., 8oz Childrens Boxing Gloves], categoryName_text_en_mv=[Sports Equipment, Outdoor], productVisible_boolean=true, url_en_string=/outdoor/sports-equipment/8oz-childrens-boxing-gloves/p/100838, pictureMap_string={min-width:1200=https://image.smythstoys.com/picture/desktop/100838.jpg, min-width:768=https://image.smythstoys.com/picture/tablet/100838.jpg, max-width:768=https://image.smythstoys.com/picture/mobile/100838.jpg}, priceValue_gbp_double=4.99, categoryNamePath_string_mv=[Outdoor, Outdoor > Sports Equipment], categoryMetaTitle_string=SM060307_Sports Equipment: Awesome deals only at Smyths Toys UK, categoryMetaDescription_string=Sports Equipment! Shop for an excellent range. Watch out for great offers at Smyths Toys UK, code_text=100838, description_text_en=8oz childrens boxing gloves., name_text_en=8oz Childrens Boxing Gloves, name_sortable_en_sortabletext=8oz Childrens Boxing Gloves, age_string_mv=[9 - 11 Years, 6 - 8 Years], categoryPath_string_mv=[/SM0603/SM060307, /SM0603], customCategoryPath_string_mv=[/curl/outdoor/c/SM0603, /curl/outdoor/c/SM0603/curl/outdoor/sports-equipment/c/SM060307], ukBestsellerRating_en_int=999999, ukBestsellerRating_sortable_en_int=999999, pictureUrl_string=https://image.smythstoys.com/picture/desktop/100838.jpg, _version_=1577946952254881793}]},facet_counts={facet_queries={},facet_fields={gender_string_mv={Female=2,Male=1},price_gbp_string={£0 - £19.99=3},categoryPath_string_mv={/SM0601=2,/SM0601/SM060103=1,/SM0601/SM060103/SM06010326=1,/SM0601/SM060104=1,/SM0601/SM060104/SM06010425=1,/SM0603=1,/SM0603/SM060307=1},allCategories_string_mv={SM0601=2,SM060103=1,SM06010326=1,SM060104=1,SM06010425=1,SM0603=1,SM060307=1},excludeFromGiftFinder_boolean={false=2,true=1},productVisible_boolean={true=3},category_string_mv={SM0601=2,SM060103=1,SM06010326=1,SM060104=1,SM06010425=1,SM0603=1,SM060307=1},brand_string_mv={Sylvanian=1,Toy Story=1},productType_string={REGULAR=3},age_string_mv={6 - 8 Years=3,3 - 5 Years=2,9 - 11 Years=2}},facet_ranges={},facet_intervals={},facet_heatmaps={}},spellcheck={suggestions={},collations={}}}
Schema.xml - Some snippets: snippet 1-
<fieldType name="text_en" class="solr.TextField" positionIncrementGap="100">
<analyzer>
<tokenizer class="solr.StandardTokenizerFactory" />
<filter class="solr.StopFilterFactory" words="lang/stopwords_en.txt" ignoreCase="true" />
<filter class="solr.ManagedStopFilterFactory" managed="en" />
<filter class="solr.SynonymFilterFactory" ignoreCase="true" synonyms="synonyms.txt"/>
<filter class="solr.ManagedSynonymFilterFactory" managed="en" />
<filter class="solr.WordDelimiterFilterFactory" preserveOriginal="1"
generateWordParts="1" generateNumberParts="1" catenateWords="1"
catenateNumbers="1" catenateAll="0" splitOnCaseChange="1" />
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.ASCIIFoldingFilterFactory" />
<filter class="solr.SnowballPorterFilterFactory" language="English" />
</analyzer>
</fieldType>
schema.xml Snippet 2:
<field name="text" type="textgen" indexed="true" stored="false" />
schema.xml Snippet 3:
<fieldType name="textgen" class="solr.TextField" positionIncrementGap="100">
<analyzer>
<tokenizer class="solr.StandardTokenizerFactory" />
<filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" />
<filter class="solr.SynonymFilterFactory" ignoreCase="true" synonyms="synonyms.txt"/>
<filter class="solr.WordDelimiterFilterFactory"
generateWordParts="1" generateNumberParts="1" catenateWords="1"
catenateNumbers="1" catenateAll="0" splitOnCaseChange="1" preserveOriginal="1"/>
<filter class="solr.LowerCaseFilterFactory" />
</analyzer>
</fieldType>
schema.xml Snippet 4:
<fieldType name="text" class="solr.TextField" positionIncrementGap="100">
<analyzer>
<tokenizer class="solr.StandardTokenizerFactory" />
<filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" />
<filter class="solr.SynonymFilterFactory" ignoreCase="true" synonyms="synonyms.txt"/>
<filter class="solr.WordDelimiterFilterFactory"
generateWordParts="1" generateNumberParts="1" catenateWords="1"
catenateNumbers="1" catenateAll="0" splitOnCaseChange="1" preserveOriginal="1"/>
<filter class="solr.LowerCaseFilterFactory" />
</analyzer>
</fieldType>
Most likely you have different results because of solr.WordDelimiterFilterFactory and the setting splitOnCaseChange=1 which will break toysChildren into toys and Children. And in case of toyschildren, there is no case change, so you will only have token toyschildren and that's exactly what make the difference in your query results.
You have several choices depends on what is expected behaviour of your system. You could turn of this setting or completely remove solr.WordDelimiterFilterFactory from fieldtype.
Related
Will SOLR perform matching on the street name?
I have a query regarding matching a street name in SOLR. The actual street name to match is POTTS ROAD EVANSFIELD VIC. I have stored the data in three fields: street_name_clean : POTTSROADEVANSFIELDVIC street_name_space : POTTS ROAD EVANSFIELD VIC street_name : POTTS, ROAD, EVANSFIELD, VIC The reason for storing the data as such is so that I can perform an exact search, fuzzy search, ngram search, proxmity matching etc. I have see a case where the user inputs POTTROAD (missing S from the actual street name) and all my searches fail. Is there a technique to match POTTROAD with the data above? Any help is appreciated.
Thanks to the suggestion from #MatsLindh, I implemented the ShingleFilter as follows: <fieldType name="text_general_shingle" class="solr.TextField"> <analyzer> <tokenizer class="solr.StandardTokenizerFactory"/> <filter class="solr.LowerCaseFilterFactory"/> <filter class="solr.ShingleFilterFactory" maxShingleSize="4" outputUnigrams="false"/> <filter class="solr.PatternReplaceFilterFactory" pattern="(\s+)" replacement="" replace="all" /> </analyzer> </fieldType>
how to force an & operator for filters in search view:
I've a default search in-progress when checking an other filter current_month the operation in the search input became in-progress or current_month
I want an AND operator as in-progress and current_month
the filters are defined as :
<filter string="in progress" domain="[('state','=','progress')]" />
<filter string="current month" domain="[('date','>=',time.strftime('%%Y-%%M-01') ) ]" />
Try to add a separator between those filters:
<filter string="in progress" domain="[('state','=','progress')]" />
<separator />
<filter string="current month" domain="[('date','>=',time.strftime('%%Y-%%M-01') ) ]" />
This is sample filter using "&" that filters to display if Assigned for the current user & with value > 1000$ Code: [[u'user_id', u'=', 1], [u'amount_total', u'>', 1000]] How to get this? The easiest way is to create a custom filter and save it in "Favourites". Then click on a ladybug and choose "Manage Filters". Once you have this copy the code and paste into your custom module.
Routing to a different channels based on condition
I would like to route the message to different channels based on the condition of the property. Let's say I have score property. If the score is <100 then it goes to "perfectchannel" else it goes to "normalchannel" Where do I specify the spel expression or condition <si:router id="serviceRouter" input-channel="serviceRoutingChannel" expression="payload.score" default-output-channel="badchannel" resolution-required="false"> <si:mapping value="100" channel="perfectchannel" /> <si:mapping value="<100 ??" channel="normalchannel" /> </si:router> Appreciate your help on this.
We have a JIRA ticket on the matter, but haven't come up with the solution yet. Right now you can achieve this behaviour with condition from the expression and providing mapping for true and false and with cascad of the routers: <si:router id="serviceRouter" input-channel="serviceRoutingChannel" expression="payload.score == 100"> <si:mapping value="true" channel="perfectChannel" /> <si:mapping value="false" channel="nestedRouterChannel" /> </si:router> <si:router input-channel="nestedRouterChannel" expression="payload.score lt 100"> <si:mapping value="true" channel="normalChannel" /> <si:mapping value="false" channel="badChannel" /> </si:router> UPDATE Another option to use <recipient-list-router>: <recipient-list-router id="serviceRouter" input-channel="serviceRoutingChannel"> <recipient selector-expression="payload.score == 100" channel="perfectchannel"/> <recipient selector-expression="payload.score lt 100" channel="normalchannel"/> <recipient selector-expression="payload.score gt 100" channel="badchannel"/> </recipient-list-router>
Use termfreq(field,term) function for phrase with space in SOLR 4.1
I am using termfreq(field,term) SOLR function. This works: ?fl=product_name,termfreq(product_name,"iphon")&q=iphone 4s //Found freq But the problem is to have term like "iphone 4s" with space ?fl=product_name,termfreq(product_name,"iphon 4s")&q=iphone 4s //Return 0 freq Return 0 freq although that term(phrase) exist in doc. So, the question is, can I use termfreq() function with full phrase like "iphone 4s", And how? I am using SOLR 4.1. and analyzer for field is <fieldType name="text_ws" class="solr.TextField"> <analyzer> <tokenizer class="solr.WhitespaceTokenizerFactory"/> <filter class="solr.LowerCaseFilterFactory"/> </analyzer> </fieldType> Field is <field name="product_name" type="text_ws" indexed="true" stored="true"/>
As you are using a WhitespaceTokenizerFactory the term iphone 4s would not exist as a term. You could use KeywordTokenizerFactory for indexing, which doesn't tokenize the words and the phrase should be available. Else you can check for shingle options which would group words for you.
Solr wrong sort text fields
I have "text_general" field in schema.xml
<fieldType name="text_general" class="solr.TextField" positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/><filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
I have stored documents:
document1:
spell = "contro un indice generale dei prezzi salito del 2, 1%. Rincari ben piщ evidenti, tra i prodotti da bar"
testata = "Mattino di Padova (Il)"
document2:
spell="con i prodotti di qualitа vinco la crisi dei consumi Farinetti: con"
testata = "Italia Oggi"
document3
spell = "convenienza Il 2008 porta i primi aumenti nei pre zi L'Ipercoop cresce il listino"
testata = "Nuova Ferrara (La)"
"spell" and "testata" fields has a "text_general" type.
Searching working fine for me:
http://localhost:8080/solr/select?q={!type=edismax qf=spell v='co*'}
But with sorting exists some problem:
http://localhost:8080/solr/select?q={!type=edismax qf=spell v='co*'}&sort=testata desc
It returns me this result:
document1:
spell = "contro un indice generale dei prezzi salito del 2, 1%. Rincari ben piщ evidenti, tra i prodotti da bar"
testata = "Mattino di Padova (Il)"
document2:
spell="con i prodotti di qualitа vinco la crisi dei consumi Farinetti: con"
testata = "Italia Oggi"
document3
spell = "convenienza Il 2008 porta i primi aumenti nei pre zi L'Ipercoop cresce il listino"
testata = "Nuova Ferrara (La)"
I don`t understand why my sorting working not properly. It should returns me result like this:
document3
spell = "convenienza Il 2008 porta i primi aumenti nei pre zi L'Ipercoop cresce il listino"
testata = "Nuova Ferrara (La)"
document1:
spell = "contro un indice generale dei prezzi salito del 2, 1%. Rincari ben piщ evidenti, tra i prodotti da bar"
testata = "Mattino di Padova (Il)"
document2:
spell="con i prodotti di qualitа vinco la crisi dei consumi Farinetti: con"
testata = "Italia Oggi"
Sorting doesn't work good on multivalued and tokenized fields. As testata has been defined with text_general field type, it will be tokensized and hence the sort would not work fine. Sorting can be done on the "score" of the document, or on any multiValued="false" indexed="true" field provided that field is either non-tokenized (ie: has no Analyzer) or uses an Analyzer that only produces a single Term (ie: uses the KeywordTokenizer) Source: http://wiki.apache.org/solr/CommonQueryParameters#sort Use string as the field type and copy the title field into the new field. <field name="testata_sort" type="string" indexed="true" stored="false"/> <copyField source="testata" dest="testata_sort" />