The problem is simple: I have written map functions in a design document of a CouchDB database, which emits something {"_id":doc._id}. Together with include_docs=true query option, I will get the desired results with the linked documents. Because the map functions are designed to work with include_docs=true, I put this option in the design document and make it default:
{...
"options":{"include_docs":true}
...}
However, when I query the view, the results are still those without the linked documents, and I need to specify the option explicitly in the query. I also tried to pu other query option (e.g. limit=200) into the design document, they did not work either.
I am using CouchDB 1.5, and cannot find any discussion, issue or bug regarding this. Does anyone have any idea? Thanks in advanced!
Edit: I have reported the issue in Apache, and I am told that the statement about this was removed.
_design/ddoc/options cannot do that.
According to couchdb's docs, a design doc's options object properties only affect view indexing, not view querying. (The only two settings being local_seq and include_design).
_design/ddoc/rewrites can!
If you want to set query options server side, you can do so by specifying a rewrites array in your design document.
Let's say you want to expose a query to _view/myview that has include_docs set to true, you add the following rewrites array to your design document:
{ "_id": "_design/myddoc"
, "views": { "myview": { "map": "function(doc) { ... }" } }
, "rewrites":
[ { "from": "allmyviews/myview"
, "to": "_view/myview"
, "query":
{ "include_docs": "true"
}
}
]
}
Now, when you request http://localhost:5984/mydb/_design/myddoc/_rewrite/allmyviews/myview without the include_docs parameter, couchdb will respond as if you had included it.
Related
I have a design document in CouchDB. I've set up views and filters.
{
"_id": "_design/my_index_id",
"_rev": "17-fa5c543fcc80f4420aa98d58f7a07130",
"views": {
"jobsbyid": {
"map": "function (doc,req) {if (doc.type === 'job') {emit(doc.id);}}"
}
},
"filters": {
"myfilter": "function (doc, req) {return req.query.type === 'job'}"
}
What's the different between views and filters. In terms of performance, use cases and usage. When to use Views and when to use filters?
In CouchDB you have different filtering options for the replication process. All of them are documented here CouchDB filtering options
About filtering, You should have in cosideration that filtering is one of the most expesive operation in CouchDB that could drive you into some performance degradation problems as long the database grows. You can check this answer Filtered Sync between CouchDB and PouchDB
The usage of filters or views are almost the same in terms of performance as they are filtering the whole database in each filtering request. This is stated in the doc
Using _view filter doesn’t queries the view index files, so you cannot
use common view query parameters to additionally filter the changes
feed by index key. Also, CouchDB doesn’t returns the result instantly
as it does for views - it really uses the specified map function as
filter.
Moreover, you cannot make such filters dynamic e.g. process the
request query parameters or handle the User Context Object - the map
function is only operates with the document.
The advantaje of the use of views for filtering is that you are reusing map functions for filtering.
So use cases of both approaches are very similar except that the filters may access to the query params or the security context.
I'd like to use a _list function to format the output of the _all_docs view.
I see a patch was merged to support this usecase, however from the docs and the comments I can't figure out what the end-point for this would be.
I've tried on Cloudant, which doesn't seem to work:
/db/_design/[design-doc]/_list/[list-name]/_all_docs
Is it the case that this is not supported on Cloudant? I don't have a CouchDB install at hand to test with.
This feature is not currently supported in Cloudant, but should be coming soon. As a workaround, you can create a custom view called _all_docs in the same design document as your list function:
…
"views": {
"_all_docs": {
"map": "function(doc) { emit(doc._id, {\"rev\": doc._rev}) }"
},
…
This will create a redundant index, which isn't ideal. This custom _all_docs also won't return design documents, where the "real" _all_docs will return design documents.
I'm using Loopback's MongoDB connector to expose a RESTful API to query and update a MongoDB collection.
When I defined the model it didn't let me define property names with spaces in them, so instead of "Gas 93" I had to use "gas93". The example is trivial but affect 9 properties of my collection.
I know you can map properties to table fields with the PostgreSQL connector
"properties": {
"name": {
"type": "String",
"required": false,
"postgresql": {
"columnName": "username",
}
}
I haven't found a way to do the same for MongoDB fieldnames. I have a lot of features in my app using collection fieldNames as they are now, so I'd like to avoid changing them to adapt to loopback.
Is there a way to do this? Documentation doesn't say it but, then again, it doesn't say you can't either.
LoopBack doesn't support property mapping for mongoDB at the moment, only collection names can be customized. We will update the documentation to clarify this.
I have added a pull request to support this:
https://github.com/strongloop/loopback-connector-mongodb/pull/189
Considering the following two documents:
{
"_id": "a6b8d3d7e2d61c97f4285220c103abca",
"_rev": "7-ad8c3eaaab2d4abfa01abe36a74da171",
"File":"/store/document/scan_bgd123.jpg",
"Commend": "Describes a person",
"DateAdded": "2014-07-17T14:13:00Z",
"Name": "Joe",
"LastName": "Soap",
"Height": "192cm",
"Age": "25"
}
{
"_id": "a6b8d3d7e2d61c97f4285220c103c4a9",
"_rev": "1-f43410cb2fe51bfa13dfcedd560f9511",
"File":"/store/document/scan_adf123.jpg",
"Comment": "Describes a car",
"Make": "Ford",
"Year": "2011",
"Model": "Focus",
"Color": "Blue"
}
How would I find a document based on multiple criteria, say for example "Make"="Ford" and "Color"="Blue". I realize I need a view for this, but I don't know what the key is going to be, and as you can see from the two documents, the key/value pairs aren't consistent. The only consistent item will be the "File" key.
I'm attempting to create couchDB database that will store the location of files, but tagged with Key/Value pairs.
EDIT:
Perhaps I should reconsider my data structure. modify it slightly?
{
"_id": "a6b8d3d7e2d61c97f4285220c103c4a9",
"_rev": "1-f43410cb2fe51bfa13dfcedd560f9511",
"File": "/store/document/scan_adf123.jpg",
"Tags": {
"Comment": "Describes a car",
"Make": "Ford",
"Year": "2011",
"Model": "Focus",
"Color": "Blue"
}
}
So, I need to find by the Key>Value pair in the tag or any number of Key>Value pairs to filter which document I want. The problem here is, I want to tag objects with a key>value pair. These tags could be very different per view, so the next document will have a whole diff set of Key>Value pairs.
Couchdb supports flexible schema. There is no need for the documents to be consistent for them to be query-able. The view for your scenario is pretty straightforward. Here is the map function that should do the trick.
function(doc){
if(doc.Make&&doc.Color)
emit([doc.Make,doc.Color],null);
}
This gives you a view which you can then query like
/view-name/key=["Ford","Blue"]&include_docs=true
This should give you the desired result.
Edit based on comment
For that you will need two separate views. Every view in couchdb is designed to fulfil a specific query need. This means that you have to think about access strategy of your data. It is a lot more work on your part initially but for the trouble you are rewarded with data that is indexed and has very fast access times.
So to answer your question directly. Create two views. One for Make like we have already done and other for Name like
function(doc){
if(doc.Name&&doc.LastName)
emit([doc.Name,doc.Name],null);
}
Now the Name view will index only those documents that have name in it. Where as Make view will index those documents that have make in it.
What happens when a requirement comes in future for which you don't have a query?
You can try a few things.
This is probably the easiest solution. Use couchdb-lucene for your dynamic queries. In this case your architecture will be like couchdb views for queries that you know your application would need. Lucene index for queries that you don't know you might need. So for instance you have indexed name and last name in the in couchdb query. But a requirement arises and you might need to query by age then simply dump the age field in lucene and it will take care of the rest.
Another approach is using the PPP technique where you exploit the fact that creating views is a one time cost and you can create views on less active hours and deploy them in a production service once they are built.
Combine steps 1 and 2! lucene to handle adhoc request while you are building views using the ppp technique.
Considering these three documents...
[
{
_id: "...",
_rev: "...",
title: "Foo",
body: "..."
},
{
_id: "...",
_rev: "...",
title: "Bar",
body: "..."
},
{
_id: "...",
_rev: "...",
title: "Hello World!",
body: "..."
},
]
And this view...
byTitle: {
map: function (document)
{
emit(document.title, document);
}
}
What goes on behind the scenes, when I query the view?...
GET /database/_design/posts/_view/byTitle?key="Foo"
I've asked a few questions on views lately... questions about what I phrased as "dynamic parameters"... Essentially I wanted to know how to do the equivalent of SELECT ... WHERE field = parameter
All answers steered me towards using temporary views, which are really slow, and should not be used in production. So my second question is... is the above method for querying by title, fit for use in production? Or am I forcing CouchDB to do unspeakable horrors, performance-wise?... am I essentially doing the same as using a temporary view?
I think you have misinterpreted some answer. You can use a temporary view to test various map/reduce functions. When you are satisfied with the code you should put it into a design document and use it for querying.
Temporary views are slow because the index is built and deleted for every query. Putting it into a design document, tells CouchDB to not delete the index and to keep it updated (this is done on query time).
So
GET /database/_design/posts/_view/byTitle?key="Foo"
is the fastest way to query by title because it is indexed.
As a side note: you can use
byTitle: {
map: function (document)
{
emit(document.title, null);
}
}
and query with include_docs=true to save some disk space.
For answering your question, a few things have to be cleared out (and I hope I get it all right):
Permanent vs. temporary views:
The difference between permanent and temporary views is, that permanent views are stored permanently.
In order to understand the storing part, you need to know, that CouchDB's storage engine relies on a B+ Tree offering very powerful indexing capabilities that enable us to find data in that storage by key in a "logarithmic amortized time" (CouchDB book).
CouchDB is handling documents in an "append only" manner. That means it is not like in the most relational DBMS where single values within a table row get updated and locking occurs. If a document is updated, it simply incrementally is set a new revision (_rev) and is appended to the storage.
When you are creating a permanent view, upon querying it the first time, for each document in your database, your new view is executed, storing that data to a new B+ tree file for that view, thus providing a new index to aggregate data according to the key you defined in your view.
Upon updating documents that are handled by that view, not the whole permanent view needs to be recomputed, but only the updated documents.
Now you should be able to understand why temporary views are nice for developing or testing in Futon, but since they have to be computed new for all your documents are not recommended for anything else than development.
Anyways. Marcello is right. If you are intending to just pass back complete documents, it is are encouraged to query with "include_docs=true". Why? Because the B-tree for your permanent view will just need to store the copied data next to your indexing key.
#Marcello-Nuccio I am not sure although if it is correct to say, that dynamic views have no index? As I understood, they have an index, but it makes no sense as they are computed new upon every query? Ok, now my brbain is hurting!