HibernateSearch/Lucene searching in index with undersclore - search

I declared a field where I wanna save values with underscores in it. Therefore I marked the field with the #Field annotation like:
#Field(name = "underscoreField", index = Index.UN_TOKENIZED).
In Luke I can see that the index is created correctly. For instance:
ABC_EF_AB
When I search for "ABC_EF_AB" or "ABC_" I can't find any result. I already tried the Standard and the Keyword Analyzer.
Thanks

Try the WhitespaceAnalyzer. It uses a WhitespaceTokenizer that breaks at whitespace, so it should allow underscores.

Related

Azure Search filter on the whole field

I've been trying to create a filter matching the end of the whole field text.
For example, taking a text field with the text: the brown fox jumped over the lazy dog
I would like it to match with a query that searches for fields with values ending with g. Something like:
{
"search":"*",
"queryType":"full",
"searchMode": "any",
...
"filter":"search.ismatchscoring('/g$/','MyField')"
}
The result is only records where MyField contains values with words composed by a the single g character anywhere on the string.
Using the filter directly also produces no results:
{
"search":"*",
"queryType":"full",
"searchMode": "any",
...
"filter":"MyField eq '*g'"
}
As far as I can see, the tokenization will always be the base for the search and filter, which means that on the above query, $ is completely ignored and matches will be by word, not by field.
Probably I could use the keyword_v2 analyzer on this field but then I would lose the tokenizarion that I use when searching normally.
One possible solution could be defining a second field in your index, with the same value as ‘MyField’, but with a different analyzer (e.g. keyword_v2). That way you may still search over the original field while filtering over the other.
Regardless, you might have simplified the filter for the sake of the example, but otherwise it seems redundant to use search.ismatchscoring() when not combining with another filter clause via ‘or’ – one can use the search parameter directly.
Moreover, regex might not be working because the default queryType for search.ismatchscoring() is simple, not full - please see docs here

How to save strings in matrixes in matlab

I want to have a matrix/cell, that has strings inside that I can access and use later as strings.
For instance, I have one variable (MyVar) and one cell (site) with names inside:
MyVar=-9999;
site={'New_York'; 'Lisbon'; 'Sydney'};
Then I want to do something like:
SitePosition=strcat(site{1},'_101'}
and then do this
save(sprintf('SitePosition%d',MyVar),);
This doesn't work at all! Is there a way to have strings in a matrix and access them in order to keep working with them if they were a string?
This:
MyVar=-9999; site={'New_York'; 'Lisbon'; 'Sydney'};
SitePosition = strcat(site{1},'_101');
save(sprintf('SitePosition%d',MyVar));
Works fine and yields SitePosition-9999.mat, note the syntax changes in lines 2 and 3.
Is there something else you're expecting?
EDIT: Based on your comment
Check out the documentation for save regarding saving specific variables
New example:
MyVar=-9999;
site={'New_York'; 'Lisbon'; 'Sydney'};
SitePosition = strcat(site{1},'_101');
save(SitePosition,'MyVar');
Creates New_York_101.mat with only the variable MyVar in it.

Numbers and special characters in search strings in Haystack+Whoosh

I'm tring to understand how haystack search works.
I hava a model Order with field Order.no where the order number is stored in a form 'ABC/2013/11/1', 'ABC/2013/11/2' ...
I want to implement autocomplete on this field using Haystack with Whoosh backend (django-haystack 2.1.0, celery-haystack 0.7.2, Whoosh 2.5.5, Django 1.6). My search_index.py looks like this:
class OrderIndex(CelerySearchIndex, indexes.Indexable):
text = indexes.CharField(document=True, use_template=True)
name_auto = indexes.EdgeNgramField(model_attr='name')
def get_model(self):
return Order
When I try
SearchQuerySet().autocomplete(name_auto='ABC/2013')
I recieve both ABC/2013/11/1 and ABC/2013/11/2 and it's ok
When I try
SearchQuerySet().autocomplete(name_auto='ABC/2013/11')
I still recieve both ABC/2013/11/1 and ABC/2013/11/2 and it's also ok
but when I try
SearchQuerySet().autocomplete(name_auto='ABC/2013/11/1')
I still recieve both ABC/2013/11/1 and ABC/2013/11/2 I don't understand why.
I also notice that when I change the number format for the whole project for '1/ABC/2013/10' ... query like
SearchQuerySet().autocomplete(name_auto='1/')
doesn't return any results and query like
SearchQuerySet().autocomplete(name_auto='1/ABC')
return both '1/ABC/2013/10' and '2/ABC/2013/10'.
Maybe I'm missing something related to numbers and/or special characters in Haystack queries/ search strings. Thanks for any help.
The reason is two-fold actually: First is that the forward-slash ("/") is a reserved character in Whoosh, so it is ignored. Second is that Whoosh also ignores single character search terms.
So your query,
'ABC/2013/11/1'
if stripped off the slashes,
'ABC 2013 11 1'
and then single characters,
'ABC 2013 11'
Looks just as if you are searching for
'ABC/2013/11' -> 'ABC 2013 11'
Funny the documentation seems to be mum about this strange behavior.

How do I pass a String into a function in an NVelocity Template?

I'm using the NVelocity Templating engine to produce a fixed-length field output - you know the kind of thing:
Field Start Pos Field Length Notes
---------- --------- ------------ ---------
Supplier 1 7 Leading Zeros
GRN 8 9 -
...
e.g.
>0001234 123A<
The problem is I'm trying to call String.PadRight() with the overload to specify the leading zero, and NVelocity is having none of it..
This works:
$Document.SupplierCode.PadRight(7)
But this doesn't:
$Document.SupplierCode.PadRight(7,"0")
I've tried:
Single Quotes ('0')
Double Single-Quotes (''0'')
Double Quotes ("0")
Double Double-Quotes (""0"")
Escaping the quotes for all of the above (\"0\")
No Quotes!
All I've found to work from is the NVelocity Homepage, and the Velocity Templating Language Reference page, niether are pointing me at a solution.
Sorry I'm unable to supply or point you somewhere where you can test out your ideas for yourself, but any suggestions you may have will be most welcome!
Thanks for your help ;o)
I'm coping with the same problem at the moment, as far as I understand it is due to the fact that PadLeft and PadRight functions of String class receive the second parameter, the leading "0", as a char, not as a string.
NVelocity allows you to specify the parameter as a string using '0', but in this way internally it generate a cast exception (or something similar), because the parameter is expected as char.
I haven't found yet (I'm just using NVelocity since 1 hour!) a way to specify the parameter as char, at the moment I have just a dirty solution such as applying a Replace(" ", "0") after the PadLeft / PadRight, so the template becomes
$Document.SupplierCode.PadRight(7).Replace(' ', '0')
One solution that a colleague has come up with is to create another property in the Document object that returns the formatted String:
E.g.
Public ReadOnly Property SupplierCodeFormatted() As String
Get
Return Supplier.Code.PadLeft(7, "0")
End Get
End Property

Invalid Parameter Name

I am having a problem with saving of data because of an incorrectly generated parameter name.
The table has a field "E-mail", and when the class wrapper is generated, the InsertCmd uses "#E-mail" as one of the parameters. In SQL Server, this is illegal and generated an exception.
I have hunted all over SubSonic for a way to modify the parameter name to simply "#Email" but the ParameterName property is read only.
I am using SubSonic 2.2 and don't have the source for it to make an internal modification.
Any ideas?
TIA
I got a mate of mine that uses SVN to pull the source code, and as expected, found a bug in the SS source.
When the column name is set in the class wrapper, the setter for the ColumnName property sets the ParamaterName property for you correctly using
"parameterName = Utility.PrefixParameter(Utility.StripNonAlphaNumeric(columnName), Table.Provider);". This removes any illegal characters like the hyphen in my E-mail column.
BUT... The property ParameterName is NOT used when the SQL commands are created. Here is the code in SQLDataProvider.GetInsertSQL, line 1462.
pars.Append(Utility.PrefixParameter( colName, this));
I changed this to
pars.Append(col.ParameterName);
and the problem is now sorted.
Thanks to you that came up with possible solutions.
You can modify the templates if you can't change the column name. See this blog post for details of how:
http://johnnycoder.com/blog/2008/06/09/custom-templates-with-subsonic/

Resources