I am new to Azure services. In my project
there is a function to search text words or multiple text words. For example, if I search "best phase", the search should return data that are related "best" and "phase" in my data.
Sample example code below. Note: searchParameters is used to sort and order my data by their date
string searchText = "best phase";
string[] temp = searchText.Contains(" ") ? searchText.Split(' ') : new string[] { searchText};
var documentSearch = _indexClient.Documents.Search("\"" + searchText + "\"^2, \"|" + searchText + "|\", +" + searchText + ", +" + string.Join(", +", temp) , searchParameters);
The current implementation consumes too much time at around 15-20 sec or more. So I need to do searches faster. Any idea how to make it faster :-)
You can use Azure Search as explained here.
You can also use Full-Text Search of SQL Azure as explained here.
To decide which one to use, please read the difference between them on this article.
Hope this helps.
Related
I got this exception - "no viable alternative at input '?'", i feel it is because of "+" query statement.
private static final String CQL_BEAN = "cql:bean:cassandraCluster";
String updataQuery = "UPDATE user_preference SET preference = preference + ? WHERE user_id = ? AND tenant_id = ? IF EXISTS";
.to(CQL_BEAN + "/" + cassandraProperties.getKeyspaceName() + "?cql=" + this.updataQuery + "&prepareStatements=false")
Update: it could be because you are using prepareStatement=false - it looks like that in this case it won't substitute placeholders... Although I'm not expert in this integration.
....
What do you want to achieve with this syntax? Update only records that were inserted previously?
Usually LWTs are used only in very limited number of situations as they require coordination between nodes in the cluster, and seriously degrade performance. More details on LWTs you can find in documentation.
We are working on integrating Solr 3.6 to an eCommerce site. We have indexed data & search is performing really good.
We have some difficulties figuring how to use Predictive Search / Auto Complete Search Suggestion. Also interested to learn the best practices for implementing this feature.
Our goal is to offer predictive search similar to http://www.amazon.com/, but don't know how to implement it with Solr. More specifically I want to understand how to build those terms from Solr, or is it managed by something else external to solr? How the dictionary should be built for offering these kind of suggestions? Moreover, for some field, search should offer to search in category. Try typing "xper" into Amazon search box, and you will note that apart from xperia, xperia s, xperia p, it also list xperia s in Cell phones & accessories, which is a category.
Using a custom dictionary this would be difficult to manage. Or may be we don't know how to do it correctly. Looking to you to guide us on how best utilize solr to achieve this kind of suggestive search.
I would suggest you a couple of blogpost:
This one which shows you a really nice complete solution which works well but requires some additional work to be made, and uses a specific lucene index (solr core) for that specific purpose
I used the Highlight approach because the facet.prefix one is too heavy for big index, and the other ones had few or unclear documentation (i'm a stupid programmer)
So let's suppose the user has just typed "aaa bbb ccc"
Our autocomplete function (java/javascript) will call solr using the following params
q="aaa bbb"~100 ...base query, all the typed words except the last
fq=ccc* ...suggest word filter using last typed word
hl=true
hl.q=ccc* ...highlight word will be the one to suggest
fl=NONE ...return empty docs in result tag
hl.pre=### ...escape chars to locate highlight word in the response
hl.post=### ...see above
you can also control the number of suggestion with 'rows' and 'hl.fragsize' parameters
the highlight words in each document will be the right candidates for the suggestion with "aaa bbb" string
more suggestion words are the ones before/after the highlight words and, of course, you can implement more filters to extract valid words, avoid duplicates, limit suggestions
if interested i can send you some examples...
EDITED: Some further details about the approach
The portion of example i give supposes the 'autocomplete' mechanism given by jquery: we invoke a jsp (or a servlet) inside a web application passing as request param 'q' the words just typed by user.
This is the code of the jsp
ByteArrayInputStream is=null; // Used to manage Solr response
try{
StringBuffer queryUrl=new StringBuffer('putHereTheUrlOfSolrServer');
queryUrl.append("/select?wt=xml");
String typedWords=request.getParameter("q");
String base="";
if(typedWords.indexOf(" ")<=0) {
// No space typed by user: the 'easy case'
queryUrl.append("&q=text:");
queryUrl.append(URLEncoder.encode(typedWords+"*", "UTF-8"));
queryUrl.append("&hl.q=text:"+URLEncoder.encode(typedWords+"*", "UTF-8"));
} else {
// Space chars present
// we split the search in base phrase and last typed word
base=typedWords.substring(0,typedWords.lastIndexOf(" "));
queryUrl.append("&q=text:");
if(base.indexOf(" ")>0)
queryUrl.append("\""+URLEncoder.encode(base, "UTF-8")+"\"~1000");
else
queryUrl.append(URLEncoder.encode(base, "UTF-8"));
typedWords=typedWords.substring(typedWords.lastIndexOf(" ")+1);
queryUrl.append("&fq=text:"+URLEncoder.encode(typedWords+"*", "UTF-8"));
queryUrl.append("&hl.q=text:"+URLEncoder.encode(typedWords+"*", "UTF-8"));
}
// The additional parameters to control the solr response
queryUrl.append("&rows="+suggestPageSize); // Number of results returned, a parameter to control the number of suggestions
queryUrl.append("&fl=A_FIELD_NAME_THAT_DOES_NOT_EXIST"); // Interested only in highlights section, Solr return a 'light' answer
queryUrl.append("&start=0"); // Use only first page of results
queryUrl.append("&hl=true"); // Enable highlights feature
queryUrl.append("&hl.simple.pre=***"); // Use *** as 'highlight border'
queryUrl.append("&hl.simple.post=***"); // Use *** as 'highlight border'
queryUrl.append("&hl.fragsize="+suggestFragSize); // Another parameter to control the number of suggestions
queryUrl.append("&hl.fl=content,title"); // Look for result only in some fields
queryUrl.append("&facet=false"); // Disable facets
/* Omitted section: use a new URL(queryUrl.toString()) to get the solr response inside a byte array */
is=new ByteArrayInputStream(solrResponseByteArray);
DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
Document doc = dBuilder.parse(is);
XPathFactory xPathfactory = XPathFactory.newInstance();
XPath xpath = xPathfactory.newXPath();
XPathExpression expr = xpath.compile("//response/lst[#name=\"highlighting\"]/lst/arr[#name=\"content\"]/str");
NodeList valueList = (NodeList) expr.evaluate(doc, XPathConstants.NODESET);
Vector<String> suggestions=new Vector<String>();
for (int j = 0; j < valueList.getLength(); ++j) {
Element value = (Element) valueList.item(j);
String[] result=value.getTextContent().split("\\*\\*\\*");
for(int k=0;k<result.length;k++){
String suggestedWord=result[k].toLowerCase();
if((k%2)!=0){
//Highlighted words management
if(suggestedWord.length()>=suggestedWord.length() && !suggestions.contains(suggestedWord))
suggestions.add(suggestedWord);
}else{
/* Words before/after highlighted words
we can put these words inside another vector
and use them if not enough suggestions */
}
}
}
/* Finally we build a Json Answer to be managed by our jquery function */
out.print(request.getParameter("json.wrf")+"({ \"suggestions\" : [");
boolean firstSugg=true;
for(String suggestionW:suggestions) {
out.print((firstSugg?" ":" ,"));
out.print("{ \"suggest\" : \"");
if(base.length()>0) {
out.print(base);
out.print(" ");
}
out.print(suggestionW+"\" }");
firstSugg=false;
}
out.print(" ]})");
}catch (Exception x) {
System.err.println("Exception during main process: " + x);
x.printStackTrace();
}finally{
//Gracefully close streams//
try{is.close();}catch(Exception x){;}
}
Hope to be helpfull,
Nik
This might help you out.I am trying to do the same.
http://solr.pl/en/2010/10/18/solr-and-autocomplete-part-1/
I have a grails domain named Person, in that i have properties: firstName, middleName, lastName and others. I do not have a fullName field in the database, its a transients property in my domain. Now I want to create a search mechanism using all those three. Is there any way to implement this?
-> Update
I thought about this from different perspective and then it hit me !
So i did this,
My query
searchResult = Person.findAll("from Person p where p.firstName LIKE '%" + searchString + "%' or p.middleName LIKE '%" + searchString + "%' or p.lastName LIKE '%" + searchString + "%'")
but when i displayed the result, i limited the number of results, so the user has to enter more specific search string to get desired result. I also added a button "See More" so if the user can see all the fetched results but by default, it would only show a limited number of results.
(I could not answer to my own question so i wrote this in the question itself :D)
As long as there is a getter method I believe the Searchable plugin will work on transients.
public String getFullName()...
you need to pass params to fetch data in limited numbers.
i.e. .findAll(Query,params)
For my website, i need to do a search mechanism, in which some of the entry field would be: Country, City, Between Dates (with or without year field), Keywords, etc etc.
My problem is, the user must decide what they wanna search for. For example, if they want to introduce just date, or date and city, or city and keyword.. etc. I dont really know how to do that, i mean, i know how to search for one thing at a time, but i'm not sure how can do this all-in-one.
a) Would i need like something like this: (if-else, if-else) and than write the code for each combination, or there is an easier way to do that?
b )Bytheway, my search mechanism is done the folowing way (i'v never done a search mechanism before, so i dont know if it is the best aproach, would apreciate some comments here also and suggestions):
class book{
String a
String b
...
Date z
String allAttributesTogether() {
a + b + c + ... + z
}
}
then in my controller, i do a double for statment and cross-match the introduced words for the search and the result of allAttributesTogether().
Thanks in advanced, VA
Check out the filter pane plugin.
When you say "search", comes to my mind search engines. But I think you are asking about querying the database, right?
If you are talking about search mechanisms, search engines are a great tool. You can take a look at Lucene, Compass, and ElasticSearch (ES) to name a few. Compass and ES are based on lucene, but are much higher in the abstraction level (easier to use).
I have been using ElasticSearch with great satisfaction.
If you are talking about querying the database, then you can just build a HQL query dynamically. The method bellow should be in a Controller, as it uses the params attribute. It is not tested ok?
List allAttributesTogether() {
def query = " select book from Book book "
def queryParams = [:]
def needsAnd = false
if(params.a || params.b || params.z ){
query += " where "
}
if(params.a){
query += " book.a = :a "
queryParams['a'] = params.a
needsAnd = true
}
if(params.b){
if(needsAnd) query += " and "
query += " book.b = :b "
queryParams['b'] = params.b
needsAnd = true
}
if(params.a){
if(needsAnd) query += " and "
query += " book.z = :z "
queryParams['z'] = params.z
}
return Book.executeQuery(query, queryParams)
}
There is also the alternative of using Criteria builder. You can also use "if" to add clauses to your Criteria clauses.
Ok,
Today I am trying to learn Subsonic. Pretty cool stuff.
I am trying to build some search functionality into my website but am struggling about how I might achieve this in Subsonic.
I have one search field that could contain multiple keywords. I want to return results that match all of the keywords. The target on the search is a single text column.
So far I have this (it runs but never returns results):
return new SubSonic.Select().From(Visit.Schema)
.InnerJoin(InfopathArchive.VisitIdColumn, Visit.VisitIdColumn)
.Where(InfopathArchive.XmlDocColumn).Like(keywords)
.ExecuteTypedList<Visit>();
There is a one to one mapping between the Visit table and the InfoPathArchive table. I just want to return the collection of Visits that have the keywords in the related XMLDocColumn.
If I could get that working it would be great. Now the second problem is that if someone searches for 'australia processmodel' then obviously the above code should only return that exact phrase. How can I create a query that splits up my search term so that it must return documents that contain ALL of the individual search terms?
Any help appreciated.
Edit: Ok, so the basic search works, but the multiple keyword search doesnt. I did what Adam suggested but it seems Subsonic only uses one parameter for the query.
Here is the code:
List<string> wordsInQueryList = keywords.Split(' ').ToList();
SqlQuery q = Select.AllColumnsFrom<Visit>()
.InnerJoin(InfopathArchive.VisitIdColumn, Visit.VisitIdColumn)
.Where(Visit.IsDeletedColumn).IsEqualTo(false);
foreach(string wordInQuery in wordsInQueryList)
{
q = q.And(InfopathArchive.XmlDocColumn).Like("%" + wordInQuery + "%");
}
return q.ExecuteTypedList();
Then if I look at the query that Subsonic generates:
SELECT (bunch of columns)
FROM [dbo].[Visit]
INNER JOIN [dbo].[InfopathArchive] ON [dbo].[Visit].[VisitId] = [dbo].[InfopathArchive].[VisitId]
WHERE [dbo].[Visit].[IsDeleted] = #IsDeleted
AND [dbo].[InfopathArchive].[XmlDoc] LIKE #XmlDoc
AND [dbo].[InfopathArchive].[XmlDoc] LIKE #XmlDoc
So it ends up that only the last keyword is being searched for.
Any ideas?
First question:
return new SubSonic.Select().From(Visit.Schema)
.InnerJoin(InfopathArchive.VisitIdColumn, Visit.VisitIdColumn)
.Where(InfopathArchive.XmlDocColumn).Like("%" + keywords + "%")
.ExecuteTypedList<Visit>();
Second question:
Pass a List of words in your query to a function that builds a SubSonic query as follows
SqlQuery query = DB.Select().From(Visit.Schema)
.InnerJoin(InfopathArchive.VisitIdColumn, Visit.VisitIdColumn)
.Where("1=1");
foreach(string wordInQuery in wordsInQueryList)
{
query = query.And(InfopathArchive.XmlDocColumn).Like("%" + wordInQuery + "%")
}
return query.ExecuteTypedList<Visit>();
Obviously this is untested but it should point you in the right direction.
You can do what Adam is suggesting or with 2.2 you can simply use "Contains()" instead of Like("%...%"). We also support StartsWith and EndsWith() :)