I created a simple view to return the blog title and time
function(doc) {
if ( doc.TITLE) emit(doc.TIME, doc.TITLE);
}
what is a simple way to display newest blog articles first (by default it is the other way around)?
Just apply descending sort order at view request e.g.
GET /dbname/_design/titles/_view/by_time?descending=True
And view output would be sorted in reversed way - newest blog articles will go first. Remember that startkey/endkey parameters will limit key range for this reversed order. More about view query parameters you could found in CouchDB wiki
Related
There's a bunch of questions on here related to pagination using CouchDB, but none that quite fit what I'm wondering about.
Basically, I have a result set ranked by number of votes, and I want to page through the set in descending order.
Here's the map for reference.
function(doc) {
emit(doc.votes);
}
Now, the problem. I found out that startkey_docid doesn't work on it's own. You have to use it in combination with startkey. The thing is, for the query, I don't use a startkey parameter (I'm not looking to restrict the results, just get the most->least). I was thinking I could just use startkey={{doc.votes}}&startkey_docid={{doc._id}} instead, but the number of votes for a document could have changed by the time someone clicks the "Next Page" link.
The way to solve this seemed obvious: just set startkey=99999999 so that it will return all documents in the database and I can just use startkey_docid to start at the one where we left off last time. Oddly, when I do that, the startkey_docid stopped working and just allowed all results to be returned again. Apparently startkey needs to exactly equal the key on the document whose _id is used in startkey_docid.
What I'm asking is whether anyone knows a workaround for using startkey_docid to page when the actual startkey could have changed by the time you want to use it? Should my application just lookup the document by _id and immediately use the doc.votes value hoping it hasn't changed in the few milliseconds between requests? Even that doesn't seem very reliable.
EDIT: Ended up switching to Mongo for the speed, so this question turned out to be kinda moot.
I have never done something like this but I think I have some idea how to do it. What you can do is to take a snapshot of the ratings and refer to it in every page. You probably want your view not to consume to much space, so you should not map separate copies of the documents with votes not changed after taking the snapshot. So, you can do the following:
Add some history of ratings with timestamp to your document.
Map the ratings AND history like this.
In your app get the current time: start_time = Date.now() and query all pages.
Cleanup the history older then the oldest active sessions.
The problem is that if you emit [votes, date] and try to paginate you will never know how many document you have to fetch to get desired number per page. There can always be some older version which you will have to skip, and you will have make next get from DB. Thats why you can consider emitting: [date, votes], read the view always twice -- for start_time and current time, and merge and sort the result (like in merge-sort).
Ad.1:
{ ...,
votes: 12,
history: [
{date: 1357390271342, votes: 10},
{date: 1357390294682, votes: 11}
]
}
Ad.2:
function (doc) {
emit([{}, doc.votes], null);
doc.history && doc.history.forEach(function(h) {
emit([h.date, h.votes], null);
});
}
Ad.3:
?startkey=[start_time, votes]&limit=items_per_page_plus1
?startkey=[{}, votes]&limit=items_per_page_plus1
Merge lists, sort by votes in your app (on in a list function).
If you will have problems with using start_docid then you can emit [date, votes, id] and query with the ID explicitly. Even when this particular doc changes its votes it will still be available in the history.
Ad.4:
If you emit [date, votes] then you can just get outdated history width: ?startkey=[0]&endkey=[oldest_active_session_time]&inclusive_end=false and update them with update handler:
function(doc, req) {
if (!doc || !doc.history) return [null, 'Error'];
var history = new Array();
var oldest = +(req.query.date);
doc.history.forEach(function(h) {
if (h.date >= oldest)
history.push(h);
});
doc.history = history;
return [doc, 'OK'];
}
Note: I have not tested it, so it is expected not to run without modifications :)
As far as I know CouchDB uses b-tree shadowing to make updates and in principle is should be possible to access older revisions of the view. I am not into the CouchDB design, so it is just a guess and there seems not to be any (documented) API for this.
I can't figure out any simple solution by now, but there are options:
Replicate not-so-often your sorting list to small dedicated db so it will be much more stale than stale=ok
Modify your schema in a way that you'll be able to sort by some more stable data. Look at the banking/ledger example in CouchDb guide: http://guide.couchdb.org/draft/recipes.html#banking. Try to log every vote and reduce them hourly for example. As a bonus you'll get a history/trends :)
I'm kind of surprised this question has been left unanswered because the functionality of CouchDB Futon basically does this when you are paginating through the results of a map function. I opened up firebug to see what was happening in the javascript console as I paginated and saw that for every set of paginated results it is passing the startkey along with startkey_docid. So although the question is how do I paginate without including startkey, CouchDB specifies that the startkey is required and demonstrates how it can work. The endkey is not specified, so if there is only one result for the specified startkey, the next set of paginated results will also contain the next key of the sorted results that do not match the startkey.
So to clarify a bit, the answer to this problem is that as you are paginating and keeping track of the startkey_docid, you also need to capture the startkey of the same document that will be the start of the next set of results. When you are calling the paginated results use both the captured startkey and startkey_docid as couchdb requires. Leave endkey off so that the results will continue on to the next key of the sorted results.
The usecase scenario for wanting to be able to paginate without specifying a key is kind of odd. So let's say that the start docid of the next paginated result did change it's key value drastically from a 9 to a 3. And we are also assuming that there is only one instance of the docid existing in the map results, even though it could potentially appear multiple times (which I believe is why the startkey needs to be specified). As the user is clicking the next button, the user's paginated results will have now moved from looking at rank 9 to rank 3. But if you are including the startkey in addition to the startkey_docid, the paginated results would just start all over at the beginning of the rank 9 results which is a more logical progression than potentially jumping over a large set of results.
Looking at the example described in Couch DB Joins.
It discusses view collation and how you can have one document for your blog posts, and then each comment is a separate document in CouchDB. So for example, I could have "My Post" and 5 comments associated with "My Post" for a total of 6 documents. In their example, "myslug" is stored both in the post document, and each comment document, so that when I search CouchDB with the key "myslug" it returns all the documents.
Here's the problem/question. Let's say I want to search on the author in the comments and a post that also has a category of "news". How would this work exactly?
So for example:
function(doc) {
if (doc.type == "post") {
emit([doc._id, 0], doc);
} else if (doc.type == "comment") {
emit([doc.post, 1], doc);
}
}
That will load my blog post and comments based on this: ?startkey=["myslug"]
However, I want to do this, grab the comments by author bob, and the post that has the category news. For this example, bob has written three comments to the blog post with the category news. It seems as if CouchDB only allows me search on keys that exist in both documents, and not search on a key in one document, and a key in another that are "joined" together with the map function.
In other words, if post and comments are joined by a slug, how do I search on one field in one document and another field in another document that are joined by the id aka. slug?
In SQL it would be something like this:
SELECT * FROM comments JOIN doc.id ON doc.post WHERE author = bob AND category = news
I've been investigating couchdb for about a week so I'm hardly qualified to answer your question, but I think I've come to the conclusion it can't be done. View results need to be tied to one and only one document so the view can be updated. You are going to have to denormalize, at least if you don't want to do a grunt search. If anyone's come up with a clever way to do this I'd really like to know.
There are several ways that you can approximate a SQL join on CouchDB. I've just asked a similar question here: Why is CouchDB's reduce_limit enabled by default? (Is it better to approximate SQL JOINS in MapReduce views or List views?)
You can use MapReduce (not a good option)
You can use lists (This will iterate over a result set before emitting results, meaning you can 'combine' documents in a number of creative ways)
You can also apparently use 'collation', though I haven't figured this out yet (seems like I always get a count and can only use the feature with Reduce - if I'm on the right track)
Document structure:
{
"Type":"post"
"LastModified":"2010-11-01 21:55",
"CategoryID":3,
"ID":12
}
Having a bunch of different post docs in different categories is great. But I can't seem to figure out how to make a view which returns the documents ordered by date, when selecting the ones in, e.g., category 3 and 5. The categories are not known, the limit query should work still.
I've tried different approaches to the view but nothing comes close to achieving the desired result.
In SQL it could probably be done by something like this:
SELECT * FROM document WHERE document.CategoryID in (3,5) ORDER BY document.LastModified DESC;
I could just query a view like this the required number of times, manually sorting and paging the data:
function(doc) {
emit(doc.CategoryID, doc.ID);
}
So does anyone know if it's possible to avoid doing that and just have couchdb be a bit smarter?
I can think of 2 possible solutions.
Emit both CategoryID and LastModified in the map function.
function(doc) {
emit([doc.CategoryID, doc.LastModified], null);
}
Now you can query the view for ?startkey=[3]&endkey=[3, {}] to get all the docs with CategoryID=3 sorted by LastModified. To get docs for multiple CategoryIDs, you need to merge the sorted results.
Use couchdb-lucene to build the index. couchdb-lucene can handle complex queries.
I have three document types MainCategory, Category, SubCategory... each have a parentid which relates to the id of their parent document.
So I want to set up a view so that I can get a list of SubCategories which sit under the MainCategory (preferably just using a map function)... I haven't found a way to arrange the view so this is possible.
I currently have set up a view which gets the following output -
{"total_rows":16,"offset":0,"rows":[
{"id":"11098","key":["22056",0,"11098"],"value":"MainCat...."},
{"id":"11098","key":["22056",1,"11098"],"value":"Cat...."},
{"id":"33610","key":["22056",2,"null"],"value":"SubCat...."},
{"id":"33989","key":["22056",2,"null"],"value":"SubCat...."},
{"id":"11810","key":["22245",0,"11810"],"value":"MainCat...."},
{"id":"11810","key":["22245",1,"11810"],"value":"Cat...."},
{"id":"33106","key":["22245",2,"null"],"value":"SubCat...."},
{"id":"33321","key":["22245",2,"null"],"value":"SubCat...."},
{"id":"11098","key":["22479",0,"11098"],"value":"MainCat...."},
{"id":"11098","key":["22479",1,"11098"],"value":"Cat...."},
{"id":"11810","key":["22945",0,"11810"],"value":"MainCat...."},
{"id":"11810","key":["22945",1,"11810"],"value":"Cat...."},
{"id":"33123","key":["22945",2,"null"],"value":"SubCat...."},
{"id":"33453","key":["22945",2,"null"],"value":"SubCat...."},
{"id":"33667","key":["22945",2,"null"],"value":"SubCat...."},
{"id":"33987","key":["22945",2,"null"],"value":"SubCat...."}
]}
Which QueryString parameters would I use to get say the rows which have a key that starts with ["22945".... When all I have (at query time) is the id "11810" (at query time I don't have knowledge of the id "22945").
If any of that makes sense.
Thanks
The way you store your categories seems to be suboptimal for the query you try to perform on it.
MongoDB.org has a page on various strategies to implement tree-structures (they should apply to Couch and other doc dbs as well) - you should consider Array of Ancestors, where you always store the full path to your node. This makes updating/moving categories more difficult, but querying is easy and fast.
I want to create pagination on application level using the CouchDB view API. The pagination uses cursors, so given a cursor, I will query the view for the n+1 documents starting with the given cursor as start key and output the n results as page and provide the n+1 result row as the cursor for the next page.
This works well as long as the view keys are also the keys for my view rows. Now this time all my docs have a date field and I emit them as map keys, because I want to sort via date. However, I can't use my cursors anymore like before.
I thought that is the reason the view API also provides startkey_docid for submitting such a cursor doc id, however this is obviously not true. It seems like this value is only applied if there are several equal rows per keys.
So, in short: I want a date-ordered view, but cursors based on the document ids. How can I do this?
Thanks in advance
Simplified view
function map(doc)
{
emit(doc.date, {_id: doc._id});
}
Simplified view result:
{
"rows":[
{"id":"123","key":"2010-06-26T01:28:13.555Z", value:{...}},
{"id":"234","key":"2010-06-22T12:21:23.123Z", value:{...}},
{"id":"987","key":"2010-06-16T13:48:43.321Z", value:{...}},
{"id":"103","key":"2010-05-01T17:38:31.123Z", value:{...}},
{"id":"645","key":"2009-07-21T21:21:13.345Z", value:{...}}
]
}
Application-level query with cursor 234, page size 3 should return:
234, 987, 103
So how can I map this to a view?
Why do you want cursors based on docid?
Map Reduce creates single dimensional indexes, so any non-key traversal will be expensive. However, I think you can do what you want without requiring traversing 2 indexes at the same time.
See for instance here how I paginate through a posts with a certain tag:
Sofa's CouchApp tag pagination
aka
http://jchris.couchone.com/sofa/_design/sofa/_list/index/tags?descending=true&reduce=false&limit=10&startkey=[%22life%22%2C{}]&endkey=[%22life%22]
The key in that view looks like ["tag","2008/10/25 04:49:10 +0000"] so you can paginate through by tag and, within tags, by time.
Edited
Ha! I just realized what you are trying to do. It is so very simple.
Forget all about docids, they should be random anyway and not related to anything so just forget docs even have ids for a second.
You say "Application-level query with cursor 234, page size 3 should return:
234, 987, 103"
Your cursor should not be 234. It should be the key "2010-06-22T12:21:23.123Z".
So in essence you use the key of the last row of results as the startkey for the next query. So eg startkey=""2010-06-22T12:21:23.123Z""&limit=3, then for each page you render, link to a query where the new startkey is the last returned key.
Bonus: with what I've just described, you will have the bottom row of page 2 be the top row of page 3. To fix this, add skip=1 to your query.
Bonus bonus: OK, what about when I have more than 3 docs that emitted to the same key in the view? Then the last key will always be the same as the first key, so you can't progress in pagination without expanding the limit parameter. Unless... you use startkey_docid (and set it do the id of the last row). That is the only time you should use startkey_docid.