Help me with understanding mongodb, please.
have three collections: threads, messages and users.
thread
{ "title" : "1212", "message" : "12121", "user_id" : "50ffdfa42437e00223000001", "date" : ISODate("2013-04-11T19:48:36.878Z"), "_id" : ObjectId("51671394e5b854b042000003") }
message
{ "message" : "text", "image" : null, "thread_id" : "51671394e5b854b042000003", "user_id" : "516d08a7772d141766000001", "date" : ISODate("2013-04-17T15:58:07.021Z"), "_id" : ObjectId("516ec68fb91b762476000001") }
user
{ "user" : "admin", "date" : ISODate("2013-04-16T08:15:35.497Z"), "status" : 1, "_id" : ObjectId("516d08a7772d141766000001") }
How can I display all messages for current thread and get user name (for comment) from users collection?
this code get only messages without user name
exports.getMessages = function(id, skip, callback) {
var skip = parseInt(skip);
messages.find({thread_id: id}).sort({date: 1}).skip(skip).limit(20).toArray(
function(e, res) {
if (e) {
callback(e)}
else callback(null, res)
});
};
Node.js and mongo native
Generally, Mongo uses embedded documents or references to maintain relationships. Here is a link from the mongo docs worth reading.
What you are currently doing is storing a manual reference to the user collection within your message collection. Mongo manual references require additional queries in order to get that referenced data. In this case, using a reference based relationship will work, but it would force the N+1 query problem. Meaning you will have to make an addition query for every message you wish to display plus the original query for messages. References are explained in further detail here. One solution would be to incorporate DBRefs, which would require language specific driver support.
Another alternative would be use embedded documents. In this case you would store the related user object embedded within the messages object. Here is another link to the mongo docs with a great example. In this case, you would make a single query, which will return all of the messages, with each related user object embedded inside. Although embedded documents encourage duplicate data, in many cases they provide performance benefits. All of this information is explained in the mongo docs and can be read in detail to further understand the data modeling of mongo.
Additionally, the mongoose library is pretty awesome and has a populate function which is helpful for references.
Related
I am working on photo sharing application that has a mongodb model that looks a bit like:
Location = { _id : id, address : address, name : name }
Photo = {
_id: id,
_upvotes : [{ ref to User }],
_location : {ref to Location},
url: url,
selected : boolean
}
A quick overview of the code that I want to modify (nodejs/expressjs/mongoose):
Photo.find({})
.populate('_location')
.stream({ objectMode : true })
.pipe(res);//res being response stream in express.js
Is there a way to query on Photo model in such a way that:
If there is no Photo.selected in a Location
Then find the photo with maximum votes, by counting the length of upvotes
I did a deep research using MongoDBs mapReduce implementation, agregation framework, and $where operator, but did not find my way out.
If there is any other question, or reference that may lead to an answer, or a specific approach to follow, please let me know.
I'm using sails-mongo in my project and i need to execute one query in an embedded collection.
My data are something like the following:
{
"_id" : ObjectId("53906c6254f36df504e99b8f"),
"title" : "my post"
"comments" : [
{
"author" : "foo",
"comment" : "foo comment"
},
{
"author" : "bar",
"comment" : "bar comment"
}
],
"createdAt" : ISODate("2014-06-05T13:10:58.365Z"),
"updatedAt" : ISODate("2014-06-05T13:10:58.365Z")
}
for example, i need to extract the comments of author foo.
Apparently sails does not support this feature yet, so i was considering using the object db of mongodb-native to make this kind of query.
As sails-mongo uses mongodb-native, can i have access to db object in my sails project? Or i will need build a new connection using mongodb-native?
If anyone has a better idea I'd be grateful. Thanks
If all you need to do is access the embedded comments, Waterline should work fine. Just do a normal find or findOne, and the comments should be accessible on the returned objects.
If you need to query the comments, e.g. to find Posts with comments by a certain author, you can access the underlying mongodb-native collection using the .native() method of your Sails model class:
Post.native(function(err, collection) {
if (err) {
// handle error getting mongo collection
}
collection.find({'comments.author':'foo'}).toArray(function(err, results) {
// Do something with results
});
});
I'm specifically talking about NodeJS with MongoDB (I know MongoDB is schema-less, but let's be realistic about the importance of structuring data for a moment).
Is there some magic solution to minimising the number of queries to a database in regards to authenticating users? For example, if the business logic of my application needs to ensure that a user has the necessary privileges to update/retrieve data from a certain Document or Collection, is there any way of doing this without two calls to the database? One to check the user has the rights, and the other to retrieve the necessary data?
EDIT:
Another question closed by the trigger-happy SO moderators. I agree the question is abstract, but I don't see how it is "not a real question". To put it simply:
What is the best way to reduce the number of calls to a database in role-based applications, specifically in the context of NodeJS + MongoDB? Can it be done? Or is role-based access control for NodeJS + MongoDB inefficient and clumsy?
Obviously, you know wich document holds which rigths. I would guess that it is a field in the document, like :
{ 'foo':'bar'
'canRead':'sales' }
At the start of the session you could query the roles a user has. Say
{ 'user':'shennan',
'roles':[ 'users','west coast','sales'] }
You could store that list of roles in the user's session. With that in hand, all that's left to do is add the roles with an $in operator, like this :
db.test.find({'canRead':{'$in':['users','west coast','sales']})
Where the value for the $in operator is taken from the user's session. Here is code to try it out on your own, in the mongo console :
db.test.insert( { 'foo':'bar', 'canRead':'sales' })
db.test.insert( { 'foo2':'bar2', 'canRead':['hr','sales'] })
db.test.insert( { 'foo3':'bar3', 'canRead':'hr' })
> db.test.find({}, {_id:0})
{ "foo" : "bar", "canRead" : "sales" }
{ "foo2" : "bar2", "canRead" : [ "hr", "sales" ] }
{ "foo3" : "bar3", "canRead" : "hr" }
Document with 'foo3' can't be read by someone in sales :
> db.test.find({'canRead':{'$in':['users','west coast','sales']}}, {_id:0})
{ "foo" : "bar", "canRead" : "sales" }
{ "foo2" : "bar2", "canRead" : [ "hr", "sales" ] }
Definitely do-able, but w/o more context it's hard to determine what's best.
One simple solution that comes to mind is to cache users and their permissions in memory so no DB lookup is required. At this point you can just issue the query for documents where permission match and...
Let me know if you need a few more ideas.
I have this SQL Database structure. A users table, a objects table and a mapping table users_objects_map to assign objects to an user account.
In SQL it works fine. With this structure it is easy to fetch objects of an user or users assigned to an object. I also can assign an object to multiple users.
users
id
firstname
lastname
...
objects
id
...
users_objects_map
user_id
object_id
What is the best way to build this with MongoDB?
My first idea was to add an array to the users where all IDs of assign objects will stored.
{"firstname":"John", "lastname": "Doe", "object_ids":["id","id2",...,"id-n"]}
But what is if a user is assigned to thousands of objects? I don't think that's a good solution. And how I'm able to fetch all users assigned to an object or all objects assigned to an user?
Is there any clever MongoDB solution for my problem?
Using object IDs within BsonArrays as a reference towards the objects is a great way to go and also consider using BsonDocuments within the "object_ids" of the user itself, then you will be able to scale it easier and using the "_id" (ObjectID) so that MongoDB indexes those IDs, this will gain performance.
Eventually you will be having 2 collections, one is with users and the other is with objects:
user:
{
"_id" : "user_id",
"firstname" : "John",
"lastname" : "Doe",
"object_ids" : [
{ "_id" : "26548" , "futurefield" : "futurevalue" },
{ "_id" : "26564" , "futurefield" : "futurevalue" }
]
}
At this moment I really don't know what kind of objects they are going to be.. but i can give you an example:
workshop object:
{
"_id>" : "user_id",
"name" : "C# for Advanced Users",
"level" : "300",
"location" : "Amsterdam, The Netherlands",
"date" : "2013-05-08T15:00:00"
}
Now comes the fun part and that is querying.
I am developing in C# and using the driver from mongodb.org.
Example:
Give me everyone that has object id == "26564".
var query = from user in userCollection.Find(Query.EQ("objects_ids._id","26564"))
select user;
This query will return the documents, in this case the users that have matched the ID.
If you have a range of values please use: Query.All("name" , "BsonArray Values");
The second query is find and/or match the IDs of the objects ID that BsonDocuments might contain.
var secondQuery =
from workshops in objectsCollection.Find(Query.EQ("_id", "userid"))
select cust["object_ids"].AsBsonArray.ToArray();
I hope I have helped you this way.
Good luck with it!
I just the beginner of couchdb so I may be misunderstand point of view so you can teach and discuss with me
Doc Type
- User
- Topic
- Comment
Requirement
- I want to webboard
- 1 Request to get this complex doc
Output I need KEY "topic-id" , VALUE {
_id : "topic-id", created_at:"2011-05-30 19:50:22", title:"Hello
World", user: {_id :
"user-1",type:"user",username:"dominixz",signature:"http://dominixz.com"}
comments: [ {_id:"comment-1", text:"Comment 1",created_at:"2011-05-30
19:50:22",user: {_id :
"user-1",type:"user",username:"dominixz",signature:"http://dominixz.com"}},
{_id:"comment-2", text:"Comment 2",created_at:"2011-05-30
19:50:23",user: {_id :
"user-2",type:"user",username:"dominixz2",signature:"http://dominixz1.com"}},
{_id:"comment-3", text:"Comment 3",created_at:"2011-05-30
19:50:24",user: {_id :
"user-3",type:"user",username:"dominixz3",signature:"http://dominixz2.com"}},
] }
I have "user" data like this
{_id:"user-1",type:"user",username:"dominixz",signature:"http://dominixz.com"}
{_id:"user-2",type:"user",username:"dominixz2",signature:"http://dominixz1.com"}
{_id:"user-3",type:"user",username:"dominixz3",signature:"http://dominixz2.com"}
"Topic" data like this {_id : "topic-id",created_at:"2011-05-30
19:50:22",title:"Hello World",user:"user-1"}
"Comment" data like this {_id:"comment-1",type:"comment" ,
text:"Comment 1", created_at:"2011-05-30 19:50:22" , user: "user-1" ,
topic:"topic-id"} {_id:"comment-2",type:"comment" , text:"Comment 2",
created_at:"2011-05-30 19:50:23" , user: "user-2" , topic:"topic-id"}
{_id:"comment-3",type:"comment" , text:"Comment 3",
created_at:"2011-05-30 19:50:24" , user: "user-3" , topic:"topic-id"}
How can I write map,reduce,list for achieve this complex data ? and how about when I wanna use LIMIT , OFFSET like in db
Thank in advance
It's a bit hard to tell what you're looking for here, but I think you're asking for a classic CouchDB join as documented in this web page.
I'd recommend reading the whole thing, but the punchline looks something like this (translated for your data):
function (doc) {
if (doc.type === 'topic') {
emit([doc._id, 0, doc.created_at], null);
} else if (doc.type === 'comment') {
emit([doc._id, 1, doc.created_at], null);
}
}
That map will return the topic ID followed by all of its comments in chronological order. The null prevents the index from getting too large, you can always add include_docs=true on your request to pull full docs when you need them, or you can use index best practices of including the bits that are interesting there.
CouchDB is a document database, not a relational database. As such it is best suited to deal with documents that encompass all the related data. While you can normalize your schema relational-style like you did, I'd argue that this isn't be best use case for Couch.
If I were to design your CMS in Couch I'd keep the topic, content and comments all in a single document. That would directly solve your problem.
You're free of course to use document stores to emulate relational databases, but that's not their natural use case, which leads to questions like this one.