I have around 53,00,000 documents in MarkLogic server and I am building a simple search application. User enters a search term and MarkLogic server searches that term in all the nodes in all the documents and returns the matching documents as the result. I have implemented a custom paging to show results per page. I am showing 10 results per page.
I am using search api for this as:-
import module namespace search="http://marklogic.com/appservices/search" at "/Marklogic/appservices/search/search.xqy";
declare variable $options:=
<options xmlns="http://marklogic.com/appservices/search">
<transform-results apply="raw"/>
</options>;
search:search($p, $options, $noRecFrom, 10)/search:result
where $p is the input from the user $noRecFrom is the number which indicates from where we have to show records. For example for page 1 $noRecFrom will be 1, for page 2 $noRecFrom will be 11, for page 3 $noRecFrom will be 21 and so on. For paging there are hyperlinks to go to First, Next, Prev and Last pages.
To calculate the total number of records returned I am using:-
for $x in search:search($p, $options)
return $x//#total;
While First, Next and Prev hyperlink works perfectly but if someone clicks Last the application stops responding and the query does not show any output. Is it due to the large number of documents in the database or I am implementing it wrongly.
Is there any efficient way for pagination in MarkLogic (for search:search) so that the user can go the Last page without delay in query result for such a large database ?
The way you've implemented it, you're running the search repeatedly in your for loop. And that would indeed be slow.
Instead, you should be calculating a $start parameter based on the #total and number of documents per page, and passing that in as an argument (I think it's the third one) to search:search.
I would also recommend making sure you can run in unfiltered mode. There is good information about optimizing for fast pagination (indexes, etc) on the developer site; the idea is to resolve queries out of indexes to give very good, accurate unfiltered performance.
If you do it that
There is a tutorial on paginated search at http://developer.marklogic.com/learn/2006-09-paginated-search
Once you have resolved the issues mentioned by cwhit above, if you still want to get to the last page of data in a faster manner, you could make your code smart enough to reverse the sort order and pull the correct offset of records.
Here's another tip:
To get better insight to what MarkLogic is doing with search:search, call
search:get-default-options()
to see the starting point for common search applications.
Related
I am building a typical pagination that allows the user to click on a particular page number and view the results (similar to the google search result view). I am using the cloudant search API for this. The cloudant search API provides the limit option but no skip option. How can I skip n number of results if the user is on page 1 and clicks on page 4 ?
I can see that the pagination is implemented using bookmarks. Does it mean that I need to first get the bookmark for page 4 by sending 3 additional requests one after another to the search api ?
There are a couple of different ways of handling this - one is the one you already suggested, which is just to fetch the pages as needed to get the bookmarks. I'm not sure there are many alternatives for search results where we can't pre-calculate the results.
Another alternative, and this depends a bit on the details of what you are trying to do, is to create a view containing the data and use the keys to narrow down the view to the results you need. View outputs support use of limit and skip which would enable you to implement pagination.
There's also a good example of pagination in the docs: http://docs.couchdb.org/en/2.1.0/ddocs/views/pagination.html
Jqgrid -Search functionality -
How to perform search locally for entire data set,without considering the pagination.
I have nearly 1500 records in my database and jqgrid has pagination .
So if I search for something,it should perform search with entire dataset irrespective of the page in the grid and it should not request database everytime ,it should be loaded only once.
Thanks In Advance
If I correctly understand your problem, then you should use lastSelectedData option, which returns, in contrast to data, filtered and sorted data items from all pages of jqGrid. The demo demonstrates the usage of lastSelectedData. It's important that the option exist in free jqGrid fork only (the fork, which I develop). If you have to use an old jqGrid version then you can follow the old answer, which describes more tricky alternative.
I have to build a search textbox in a web page similar to facebook search box. Client side there will be ajax calls. The user need to search into around 300.000 elements that have a description of a few words or an alphanumeric code. When user enters the beginning of a word, a call is made to the server which return best match based on the starting of any word or code but also suggest first the elements most recently by the user, then by the group the user belongs to and finally from the entire set. Result can be limited to 10-20 items.
How can I build a fast search by key with the value just the description of the element? We use SQL server but any other DB could be OK.
The implementation at the time was very complex to summarise here but I came across recently to UI-select that solve the front end problem nicely and it's very good component if you are using Angular
https://github.com/angular-ui/ui-select
then backend you can put whatever you have (I did with Redis)
I'm starting to use the Google Custom Search Engine in order to retrieve a temporal use of some selected word in an online newspaper.
I see that for example my result provides a total of 22000 retrieved articles. I tried to retrieve pages after the 100 index but I can't get any result.
I also tried to search directly on the google web page, but I see that after the 10 page I can't go further, so this only show me the first 1000 result at max.
Does it is possible to retrieve every single result or I've to get just only a small portion of that?
Thanks
I am implementing a solution in Solr where I have a lot of values in my facet.
As opposed to displaying a long list of values(facets) down the side of my page I want to display the top 10. And also have one for other.
For instance I would be faceting on Nationality.
So, I do not want to have a list of every nationality, Nor do I want a "see all" button.
What I require is the top 10 nationalitys and then "Other".
When a user clicks on other, it facets on this?
This is quite easy in Solr.. All you need to do is add a
&facet.limit=10
e.g.
http://solrserver:8080/solr/select&version=2.2&q=solr&start=0&rows=0&indent=on&facet=on&facet.field= nationality&facet.limit=10
to your request and you should be able to limit the results.
For more information you can check out my blog post on faceting in solr:
http://www.craftyfella.com/2010/01/faceting-and-multifaceting-syntax-in.html
or the solr wiki here:
http://wiki.apache.org/solr/SimpleFacetParameters#facet.limit