Solr stopwords magic - search

Question

My stopwords don't works as expected.
Here is part of my schema:
<fieldType name="text_general" class="solr.TextField">
<analyzer type="index">
<tokenizer class="solr.KeywordTokenizerFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.KeywordTokenizerFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
<fieldType class="solr.TextField" name="text_auto">
<analyzer type="index">
<charFilter class="solr.HTMLStripCharFilterFactory"/>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="false"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
<filter class="solr.ShingleFilterFactory" maxShingleSize="3" outputUnigrams="true" outputUnigramsIfNoShingles="false"/>
</analyzer>
<analyzer type="query">
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="false"/>
</analyzer>
</fieldType>
<field name="deal_title_terms" type="text_auto" indexed="true" stored="false" required="false" multiValued="true"/>
<field name="deal_description" type="text_general" indexed="true" stored="true" required="false" multiValued="false"/>
In stopwords.txt I have next words: the, is, a;
Also I have next data in my fields:
deal_description - This is the my description
deal_title_terms - This is the deal title a terms (will be splitted in terms)
When I try to search deal_description:
Example 1: "deal_description: his is the m" - I expect that document with deal_description "This is the my description" will be returned
Example 2: "deal_description: is th" - I expect that nothing will be found because "is" and "the" are stopwords.
When I try to search deal_title_terms:
Example 1: "deal_title_terms: is" - I expect that nothing will be found because "is" is stopword.
Example 2: "deal_title_terms: is the deal" - I expect that "is" and "the" will be ignored and term "deal" will be found.
Example 3: "deal_title_terms: title a terms" - I expect that "a" will be ignored and term "title terms" will be found.
Question 1: Why stopwords don't works for "deal_description" field ?
Question 2: Why for field "deal_title_terms" stopwords not removed for my query ?(When I am trying to find title a terms it will not find "title terms" term)
Question 3: Is there any way to show stopwords in search result but prevent them from searching ? Example:
data: This is cool search engine
search query : "is coo" -> return "This is cool search engine"
search query : "is" -> return nothing
search query : "This coll" -> return "This is cool search engine"
Question 4: Where I can find detailed description (maybe with examples) how stopwords works in solr ? Because it looks like magic.

Answer 1

Answer to Question 1 : Replace the "KeywordTokenizerFactory" as it does no actual tokenizing, so the entire input string is preserved as a single token.Use StandardTokenizerFactory instead.
Or use the below fieldType.
<fieldType name="text_general" class="solr.TextField" positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true"/>
<filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
Stopwords will work as expected for the "deal_description" field.
Answer to Question 3 : Yes. Add the StopFilterFactory in analyzer of type="query" only. It will prevent them from searching and not adding them while indexing.
Answer to Quesion 4 : https://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters
Answer to Quesion 2 : The custom field created by you seems incorrect. The text has to tokenised first using the tokenizers but you are using filters first.
Check the analysis of it with solr analysis page.