Numerous readings indicate that skip should REALLY be avoided when doing pagination. As in the link most cite that using startKey and limit is the way to go. After I get the first page I know the startKey of that page, the lastKey of that page and the total entries. If I have a pagination control with page numbers as buttons and the user selects page 3, how do I get there? I have no idea what the startKey of page3. Perhaps, I do a simple view up front to go get the start keys for the each page.
This page nicely describes pagination:
http://docs.couchdb.org/en/1.6.1/couchapp/views/pagination.html
So, you can't really have a "Go to page 298", but to have links to the previous and next 5 pages, you can look up a larger number of preceeding and following documents and generate links accordingly. For example, if you have 10 posts per page, look up 50 following keys and take every 10th one.
As for making a "Go to page X", perhaps a background script that generates somesort of cache?
Related
I am building a typical pagination that allows the user to click on a particular page number and view the results (similar to the google search result view). I am using the cloudant search API for this. The cloudant search API provides the limit option but no skip option. How can I skip n number of results if the user is on page 1 and clicks on page 4 ?
I can see that the pagination is implemented using bookmarks. Does it mean that I need to first get the bookmark for page 4 by sending 3 additional requests one after another to the search api ?
There are a couple of different ways of handling this - one is the one you already suggested, which is just to fetch the pages as needed to get the bookmarks. I'm not sure there are many alternatives for search results where we can't pre-calculate the results.
Another alternative, and this depends a bit on the details of what you are trying to do, is to create a view containing the data and use the keys to narrow down the view to the results you need. View outputs support use of limit and skip which would enable you to implement pagination.
There's also a good example of pagination in the docs: http://docs.couchdb.org/en/2.1.0/ddocs/views/pagination.html
I have around 53,00,000 documents in MarkLogic server and I am building a simple search application. User enters a search term and MarkLogic server searches that term in all the nodes in all the documents and returns the matching documents as the result. I have implemented a custom paging to show results per page. I am showing 10 results per page.
I am using search api for this as:-
import module namespace search="http://marklogic.com/appservices/search" at "/Marklogic/appservices/search/search.xqy";
declare variable $options:=
<options xmlns="http://marklogic.com/appservices/search">
<transform-results apply="raw"/>
</options>;
search:search($p, $options, $noRecFrom, 10)/search:result
where $p is the input from the user $noRecFrom is the number which indicates from where we have to show records. For example for page 1 $noRecFrom will be 1, for page 2 $noRecFrom will be 11, for page 3 $noRecFrom will be 21 and so on. For paging there are hyperlinks to go to First, Next, Prev and Last pages.
To calculate the total number of records returned I am using:-
for $x in search:search($p, $options)
return $x//#total;
While First, Next and Prev hyperlink works perfectly but if someone clicks Last the application stops responding and the query does not show any output. Is it due to the large number of documents in the database or I am implementing it wrongly.
Is there any efficient way for pagination in MarkLogic (for search:search) so that the user can go the Last page without delay in query result for such a large database ?
The way you've implemented it, you're running the search repeatedly in your for loop. And that would indeed be slow.
Instead, you should be calculating a $start parameter based on the #total and number of documents per page, and passing that in as an argument (I think it's the third one) to search:search.
I would also recommend making sure you can run in unfiltered mode. There is good information about optimizing for fast pagination (indexes, etc) on the developer site; the idea is to resolve queries out of indexes to give very good, accurate unfiltered performance.
If you do it that
There is a tutorial on paginated search at http://developer.marklogic.com/learn/2006-09-paginated-search
Once you have resolved the issues mentioned by cwhit above, if you still want to get to the last page of data in a faster manner, you could make your code smart enough to reverse the sort order and pull the correct offset of records.
Here's another tip:
To get better insight to what MarkLogic is doing with search:search, call
search:get-default-options()
to see the starting point for common search applications.
I'm creating a page system using CouchDB, showing:
10 items per page
a link to the previous page (if any)
a link to the next page (if any)
From this article on the topic, I understand that using skip is suboptimal, and that I should instead use the startkey property to specify the first document, read 11 documents from there, display the first 10 and use the key of the 11th to display the link to the next page. What troubles me is the link to the previous page. The article says:
Populating the link to the previous page is as simple as carrying the current startkey over to the next page. If there’s no previous startkey, we are on the first page.
This works when going to the next page: when I move from page 4 to page 5 I can remember that the previous page was 4. But when I move back from page 5 to page 4, I have no way of carrying over the startkey of page 3. How can this work?
Is it possible (and recommended) to use endkey along with skip=10 and limit=1 to find the first element on the previous page, so that I may create a link back to it?
You can in fact only ask for 11 documents with no skip, and that is what Futon does (look at CouchDB logs).
The trick
Both the next and previous page link will be similar: startkey is the first or last element, with a skip=1 to avoid overlapping. You then have to correctly use the descending parameter to get previous documents or next documents.
The execution
Whenever you're asking for a page, CouchDB answers with eleven documents. Let's say the key of the first one is first and the key of the last one is last. The pagination links will look like:
"next": /db/_view/myview?descending=true&limit=11&startkey=last&skip=1
"back": /db/_view/myview?descending=false&limit=11&startkey=first&skip=1
Et voilà! You just have to reverse the documents before displaying them when descending is false. ("Finding your data with views" from the CouchDB guide explains nicely the relation between those parameters and B-Trees.)
Bonus
You can easily get the docid of the first or last page (limit=1 and descending true or false), and get a pagination system that looks a lot like something you would have with a classical database (first, last, previous, next).
Read 21 documents instead of 11 - One extra going forward, and ten going backwards. The very first one holds the key to the previous page.
I am implementing a solution in Solr where I have a lot of values in my facet.
As opposed to displaying a long list of values(facets) down the side of my page I want to display the top 10. And also have one for other.
For instance I would be faceting on Nationality.
So, I do not want to have a list of every nationality, Nor do I want a "see all" button.
What I require is the top 10 nationalitys and then "Other".
When a user clicks on other, it facets on this?
This is quite easy in Solr.. All you need to do is add a
&facet.limit=10
e.g.
http://solrserver:8080/solr/select&version=2.2&q=solr&start=0&rows=0&indent=on&facet=on&facet.field= nationality&facet.limit=10
to your request and you should be able to limit the results.
For more information you can check out my blog post on faceting in solr:
http://www.craftyfella.com/2010/01/faceting-and-multifaceting-syntax-in.html
or the solr wiki here:
http://wiki.apache.org/solr/SimpleFacetParameters#facet.limit
What's a good way to paginate a list that is constantly growing?
For example, a user may open page 2 of some listing, keep looking at it for a while and then open page 3. Meanwhile, new records have been added to the beginning of the list (page 1) and caused page 3 to contain items that could have been on page 2 previously.
Should such a list be paginated from the other end? Then how to deal with the front page and the fact that it would, without special attention, contain TOTAL NUMBER OF ITEMS % PAGE SIZE items (ie. not constant)?
Assuming you have a fixed page size, some specified ordering, and the user is specifying the page they wish to view, I would simply fetch the data accordingly. Trying to do anything more complicated than that will just end up causing you unnecessary headache and confuse the user.
I think the element you're missing here is the ordering. If you specify an ordering then the user will intuitively understand. Also, that's how the majority of pagination is done, so you're not deviating from what you're user really expects.
I'd add a warning to advise the user that new items have come in and allow them to refresh the list. If they choose not to then maintain list in the state it was in when they clicked on the "Page 2" button. You can use a timestamp on the URL to determine which version of the list to serve to which user.
I do not think it's a good idea to page through a growing list. You'd better recalculate items that should be displayed on a page everytime user performs an action.