I have a User schema with basic fields which include interests, location co-ordinates
I need to perform POST request with a specific UserId to get the results
app.post('api/users/search/:id',function(err,docs)
{ //find all the documents whose search is enabled.
//on documents returned in above find the documents who have atleast 3 common interests(req.body.interests) with the user with ':id'
// -----OR-----
//find the documents who stay within 'req.body.distance' compared to location of user with':id'
//Something like this
return User
.find({isBuddyEnabled:true}),
.find({"interests":{"$all":req.body.interests}}),
.find({"_id":req.params.id},geoLib.distance([[req.body.gcordinates],[]))
});
Basically i need to perform find inside find or Query inside query..
As per your comments in the code you want to use multiple conditions in your find query such that either one of those condition is satisfied and returns the result based on it. You can use $or and $and to achieve it. A sample code with conditions similar to yours is given below.
find({
$or:[
{ isBuddyEnabled:true },
{ "interests": { "$all":req.body.interests }},
{ $and:[
{ "_id":req.params.id },
{ geoLib.distance...rest_of_the_condition }
]
}
]
});
Related
I have a collection in MongoDB with more than 5 million documents. Whenever I create a document inside the same collection I have to check if there exists any document with same title and if it exists then I don't have to add this to the database.
Example: here is my MongoDB document:
{
"_id":ObjectId("3a434sa3242424sdsdw"),
"title":"Lost in space",
"desc":"this is description"
}
Now whenever a new document is being created in the collection, I want to check if the same title already exists in any of the documents and if it does not exists, then only I want to add it to the database.
Currently, I am using findOne query and checking for the title, if it not available only then it is added to the database. I am facing the performance issue in this. It is taking too much time to do this process. Please suggest a better approach.
async function addToDB(data){
let result= await db.collection('testCol').findOne({title:data.title});
if(result==null){
await db.collection('testCol').insertOne(data);
}else{
console.log("already exists in db");
}
}
You can reduce the network round trip time which is currently 2X. Because you execute two queries. One for find then one for update. You can combine them into one query as below.
db.collection.update(
<query>,
{ $setOnInsert: { <field1>: <value1>, ... } },
{ upsert: true }
)
It will not update if already exists.
db.test.update(
{"key1":"1"},
{ $setOnInsert: { "key":"2"} },
{ upsert: true }
)
It looks for document with key1 is 1. If it finds, it skips. If not, it inserts using the data provided in the object of setOnInsert.
I hope I don't miss something obvious, but I haven't seen anywhere a syntax like that.
I'm working for an online courses provider where the user subscribes by topic.
A course can have multiple topics
User must have subscribed for all topics of that course to have access to it
I know how to make a request like { topics: { $nin : ...allTheTopicsUserHaveNoAccessTo } } but that forces me to get all "not allowed" topics first.
Is there a way to make that kind of request in one call ?
User model:
const userModel = {
allowedTopics: [ 'math', 'physics' ],
}
Course model:
const courseModel = {
topics: [ 'math', 'physics', 'biology' ], // user need to have subscribed to those three to see it
}
setIsSubset does it in a single command and can be used in a regular find with $expr operator:
db.course.find({
$expr: {
$setIsSubset: [
"$topics",
user.allowedTopics
]
}
})
https://mongoplayground.net/p/JfSiSrboOuj
It still won't let you benefit from multikey indexes though.
Yes it can be done.
Aggregation with $setDifference followed by a match for empty array will accomplish that.
However, such an aggregation pipeline would not be able to make use of an index, and would have to fetch from disk and examine every course in the entire catalog for every single run.
I am building a platform for students to give mock test on. Once the test is complete, a results are to be generated for them relative to other students who attempted the said test.
Report contains multiple parameters i.e. rank, rank within their batch, and stuff like average marks people got on the given test are updated.
To get each of this data, I need to perform a separate query on the database and then I got to update 1. the result of current user who attempted the test 2. the result of everyone else (i.e. everyone's rank changes on new attempts)
So I need to perform multiple queries to get the data and run 2-3 update queries to set the new data.
Given mongodb calls are asynchronous, I can't find a way to gather all of that data at one place to be updated.
One way is to put the next query within the callback function of the previous query but I feel like there should be a better way than that.
Maybe you could use Promise.all()
Example:
const initialQueries = []
initialQueries.push("Some promise(s)")
Promise.all(initialQueries).then(results => {
// After all initialQueries are finished
updateQueries()
}).catch(err => {
// At least one failed
})
Use db.collection.bulkWrite.
It allows multiple document insertions, updates (by _id or a custom filter), and even deletes.
const ObjectID = require('mongodb').ObjectID;
db.collection('tests').bulkWrite([
{ updateOne : {
"filter" : { "_id" : ObjectID("5d400af131602bf3fa09da3a") },
"update" : { $set : { "score" : 20 } }
}
},
{ updateOne : {
"filter" : { "_id" : ObjectID("5d233e7831602bf3fa996557") },
"update" : { $set : { "score" : 15 } }
}
}
]);
New in version 3.2.
I have a collection in my database that contains a field which is composed of 3 arrays, like this :
use_homepage: {
home: [Array],
hidden: [Array],
archive: [Array]
}
This field represents the homepage of a user.
Each array contains an ObjectID that identifies projects shown on the user homepage.
I would like to check if my project id is in use_homepage.home or use_homepage.hidden, and if it is, remove the id from the array that match.
Can I do this with 1 (or 2 max) requests or do I have to make a request each time I have to check in another array ?
In case you expect to update one document at most, you can try this:
db.entities.findAndModify({
query: { $or : [
{ home: ObjectId('<HERE YOUR ID TO BE FOUND>') },
{ hidden: ObjectId('<HERE YOUR ID TO BE FOUND>') }
]},
update: { $pull: {
home: ObjectId('<HERE YOUR ID TO BE DELETED>'),
hidden: ObjectId('<HERE YOUR ID TO BE DELETED>')
}
}
});
As you can see, in general, you can search for some value and delete some other value.
The statement returns the original matching document (i.e. before the deletion is performed). If you want the modified document you can add the following attribute:
new: true
In case you search for many documents to update, this solution does not work, since findAndModify() works just on the first document matching the query condition.
Finaly, i used to make 2 requests to do the job :
db.User.find({"use_homepage.home": id}, {_id: 1}).toArray(function(err, result) {
// If some users have the id in the array home
db.User.updateMany({_id: {$in: users_match_ids}}, {
$pull: {"use_homepage.home": id}
}
});
// Do the same with 'hidden' array
If anyone see this post and have a better solution, I take it :)
I have a collection with feeds. The documents are structured something like this:
{
_id: '123',
title: 'my title',
openedBy: ['321', '432', '543'] // ID of users
}
Then I have users:
{
_id '321',
friends: ['432'] // ID of users
}
What I would like to accomplish is to get the number of friends that has opened the feeds fetched by the user. I do this now with a mapReduce, passing the friends of the user fetching the feeds. I do not think I am doing it correctly as I reduce by only returning the emit itself and I have to convert the result back to a normal query result on the finalizer:
db.collection(collectionName).mapReduce(function () {
var openedByFriendsLength = 0;
for (var x = 0; x < friends.length; x++) {
if (this.openedBy.indexOf(friends[x]) >= 0) {
openedByFriendsLength++;
}
}
emit(this._id, {
title: this.title,
openedByLength: this.openedBy.length,
openedByFriendsLength: openedByFriendsLength
});
}, function (key, emits) {
return emits[0];
}, {
out: 'getFeeds',
scope: {
friends: user.friends
},
}, function (err, collection) {
collection.find().toArray(function (err, feeds) {
// Convert the _id / value to a normal find result
var resultFeeds = [];
for (var x = 0; x < feeds.length; x++) {
resultFeeds.push(feeds[x].value);
resultFeeds[resultFeeds.length - 1]._id = feeds[x]._id;
}
callback(err, resultFeeds);
});
});
I have looked at aggregation, but I can not quite figure out how to do the same thing. Or is the structure of the documents here all wrong?
Thanks for any reply!
You ask how to do the calculation using the aggregation framework. In general the aggregation framework performs better than map-reduce. You can find documentation on the Aggregation Framework here: http://docs.mongodb.org/manual/aggregation/.
I understand that the calculation you want is, given a user, to find all feeds where that user is contained in the openedBy array, and then find the number of distinct friends of that user that are contained in those openedBy arrays. Do I have that correct?
Aggregation, like map-reduce, only operates on one collection at a time, so the first step is to obtain the list of friends for the user from the users collection, for example:
friends = db.users.findOne({_id:user}).friends
Then we can perform the following aggregation on the feeds collection to do the calculation:
db.feeds.aggregate([
{$match: {openedBy: user}},
{$unwind: '$openedBy'},
{$match: {openedBy: {$in: friends}}},
{$group: {_id: '$openedBy'}},
{$group: {_id: 0, count: {$sum: 1}}}
])
The aggregate command specifies a list of processing steps that work much like a Unix pipeline, passing streams of documents from one stage of the pipeline to the next.
The first first step in the pipeline, $match, takes as input all documents in the collection and selects only those where the user is contained in the openedBy array.
The second step, $unwind, takes each input document and produces multiple output documents, one for each member of the openedBy array; each output document contains an openedBy field whose value is a single user. These will be users that opened the same feeds as the given user. This step will allow later steps of the pipeline to perform aggregation operations on the indivdual values of the openedBy array.
The third step, $match, filters those documents to pass only the ones where the openedBy user is a friend of the given user. However a given friend may be represented more than once in this stream, so aggregation will be needed to eliminate the duplicates.
The fourth step, $group, performs an aggregation, generating one output document for each value of the openedBy field. This will be the set of unique friends, without duplication, of the given user who have opened a feed that the user opened. The _id field will be the friend user id.
The final step, another $group, counts the number of documents generated by the preceding step. It outputs a single document, with an _id of 0 (you could use any value you want here), and with a count field that contains the final count that you wished to calculate, for example:
{ "result" : [ { "_id" : 0, "count" : 2 } ], "ok" : 1 }
I hope this answer is helpful! Let me know if you have further questions.
Bruce