Learning Map-Reduce in MongoDB - node.js

While searching through the internet, I found out that joins can be emulated in mongodb through the map-reduce function. Going through the docs was confusing.
I have two collections: one with a list of friends of one user. And the other collection is of all the users. I want to fetch the profile pictures of all the friends. how do I create a mongodb query to get the desired results?
The USERS collection:
{
"_id" : ObjectId("524c194a6e3715ce0a000001"),
"email" : "qwerty#abc.com",
"password" : "",
"phone" : "",
"salt" : "",
"upic" : "someuser2fd2751259bb7519d7b760ffee9b7fce203ad1f34.jpg",
"username" : "someuser2"
}
{
"_id" : ObjectId("524be475fafb35480a000001"),
"email" : "",
"password" : "",
"phone" : "",
"salt" : "",
"upic" : "amitverma2522b7a52e054c350f78fd7f3558919f2e2dab58.jpg",
"username" : "amitverma"
}
The friends of each user collection:
{
"_id" : ObjectId("526547ed2389630000000001"),
"friends" : [
{
"_id" : ObjectId("524be475fafb35480a000001"),
"username" : "amitverma"
},
{
"_id" : ObjectId("524be475fafb35480a000001"),
"username" : "someuser2"
}
],
"upic" : "macbookfd2751259bb7519d7b760ffee9b7fce203ad1f34.jpg",
"username" : "someuser"
}
Help would be appreciated.

There are no official docs for this as it's not a recommended best practice. It's complex that you need to do multiple passes carefully outputting the same results into the same collection.
You'd be better served by gathering the list of friends and using the $in operator (reference) to fetch the users and projecting the results to only include the fields you require (like the image).
Ideally, you'd cache those results locally to avoid needlessly requesting image paths. Following is untested code that should work in the shell:
db.friends.find({ username: 'someuser'}).forEach(friend_list) {
// this would gather the list of friend's _ids
// the _id will be passed as an array for the $in operator
var friends = friend_list.friends.map(function(friend) { return this._id; });
// gather up the images for each of the friends
var upics = db.users.find({_id : { $in : friends }},
{ _id: 1, upic: 1 }).toArray();
// now, do something with upics -- outside of the MongoDB shell, this will
// return asynchronously ....
});

Related

mongoose return null in the query

my mongodb has a collection with data like this
{
"_id" : ObjectId("62ead2a8dd6922cfd6f466e4"),
"t" : "d",
"u" : {
"_id" : ObjectId("621d3469dd01e282b9a62321"),
"username" : "helxsz"
},
"users" : [
ObjectId("621d3469dd01e282b9a62321"),
ObjectId("628ee99ed0a58e00496a0730")
],
"createdAt" : ISODate("2022-08-03T19:55:20.965Z"),
"updatedAt" : ISODate("2022-08-03T19:55:20.965Z")
}
I am using node.js and mongoose to query the document.
let query = {
u:{
_id: "621d3469dd01e282b9a62321",
username: "helxsz"
},
t:'d',
};
collection
.findOne(query, 'u t ')
.exec(getResult);
why the executed query returns null to me
Maybe is because in your DB the u._id is ObjectId and in your query is a string. Mongoose should (?) parse it but I've faced not-parsed error like this many times.
So try parsing to ObjectId, in this example works.
Also an other problem is trying to search an object like this:
{
u:{
_id: "621d3469dd01e282b9a62321",
username: "helxsz"
}
}
Because in this way mongo looks for by the objects with the same order. You have to use dot notation
As an example, check how this query not work all times. To ensure the result you have to use dot notation:
let query = {
"u._id": "621d3469dd01e282b9a62321", // maybe casting to ObjectId is necessary
"u.username": "helxsz",
t: "d"
}
Example here
More info about Match an Embedded/Nested Document

Firebase Realtime Database Query: Filtering data based on the UserId signed in and a further unique identifier within the database

I have a query regarding an app I am trying to develop with node js react and Firebase Realtime Database.
The app is for a school and I am trying to write the correct code for filtering the data by course based on the course that the student has signed up for.
On the Firebase realtime database, I have two structure as per below:
- Courses
{
"courseData" : [ {
"course" : {
"day" : "Tuesday",
"duration" : "10 weeks",
"language" : "German",
"location" : "Online",
"startdate" : "12th January",
"term" : "January",
"time" : "17.30-18.30",
"timeofday" : "Evening",
},
"courseID" : "JRNGETNXXOLTUV",
"dates" : {
"class1" : "12/01/2021",
}
}],
"users" : {
"kwvjUSgZKXXfxxxxxxxxxxxxxxxxxx" : {
"courseID" : "JRNGETNXXOLTUV",
"email" : "test#test.com",
"username" : "Test"
},
"vXf4WcRGQcxxxxxxxxxxxxxxxxxxx" : {
"courseID" : "JRNGETNXXOLTUV",
"email" : "test2#test.com",
"username" : "Test Test"
I have a courseID in both courseData and the users section of the Firebase Realtime Database.
At the moment I can generate course data for a specific course when I manually insert the courseID as you will see below in the excerpt below.
Excerpt 1
filtercourse(courseID) {
return function (coursedata) {
return coursedata.courseID === courseID;
};
}
....
Excerpt 2
<tbody>
{this.state.courseData.filter(this.filtercourse('JANSPADBGOLWEE')).map((data, index) => (
<tr key={index}>
...
Instead of manually inserting the courseID (in this case it's JANSPADBGOLWEE), I understand that I need to create a function where the courseData data is filtered by course/ courseID based on the courseData.courseID being equal to the users.uid.courseID, however, I this is beyond me it seems. Any help or advice here would be greatly appreciated.
I think you're looking for a Firebase query, which allow you to sort and filter data.
On Courses, you could get only its child nodes where courseID has a specific value with:
let courses = firebase.database().ref().child("courseData");
let courseSuery = courses.orderByChild("courseID").equalTo("JANSPADBGOLWEE");
courseSuery.once("value").then((snapshot) => {
snapshot.forEach((courseSnapshot) => {
console.log(courseSnapshot.key, courseSnapshot.child("course/day").val());
});
});
If you first need to look up the course ID for the current user, that'd be:
let users = firebase.database().ref().child("users");
if (!firebase.auth().currentUser) throw "No current user";
let uid = firebase.auth().currentUser.uid;
users.child(uid).once("value").then((userSnapshot) => {
console.log(userSnapshot.val().courseID);
});
Note that this is a fairly standard way of loading data from Firebase, so I recommend reading some more of the documentation, and taking a few tutorials to get self-sufficient with it.

mongoose $match wont return document

I use two ways to retrieve documents from my collection, the first one:
db.comments.find({"nid" : "req.body.data"});
returns many doc like:
{
"nid" : 20404,
"_id" : ObjectId("5638ba331294943d3d0a092b"),
"uid" : 1937,
"posted" : ISODate("2015-11-03T13:44:19.811Z"),
"text" : "txt",
"title" : "Test nid 2",
"stars" : 3,
"__v" : 0
}
,
And for another query I need to use aggregate and the query:
var pipleline = [
{$match: {nid:req.body.data}}
];
Comments.aggregate(pipleline, function(err, rank){
if(err) {
res.send("Error", String(err));
}
res.send(rank);
});
Returns [] - empty array.
Any ideas?
You can use the built in function chaining mongoose provides. Aside from match, it also has sort, project, group, and few others I don't know off the top of my head. More info here
Comments.aggregate().match({nid:req.body.data})
.exec(function(err,rank){
if(err) {
res.send("Error", String(err));
}
res.send(rank);
});

selecting an nested attribute to update in mongo with node driver

In the following code using the Node Js Driver for MongoDB, the console log in the callback would log the number of drivers for a particular vehicle. My problem is with trying to increment the number of drivers by one with this code
$inc:{vehicles[vehicle_number].drivers:1}
It's giving me unexpected token errors with the [ and also I'm not even sure if by starting the selector with vehicles it would be acting on the family that's been queried. Can you explain how I might change the code to make the increment not break the function.
families.findAndModify({_id:family_id}, [],
{$inc:{vehicles[vehicle_number].drivers:1}} , function(err, doc) {
console.log(doc.vehicles[vehicle_number].drivers)
})
Update
the vehicles key contains an array of vehicles. In the code above, vehicle_number represents the index in the array. I'm trying to increment the number of drivers in the findAndModify above.
{
"_id" : ObjectId("5143ddf5bcf1bf4ab37da054"),
"name" : "Jones",
"vehicles" : [
{
"make" : "Ford",
"year" : "2001",
"color" : "blue",
"registration" : "xdklde",
"drivers" : 3
},
{
"make" : "Dodge",
"year" : "1992",
"color" : "green",
"registration" : "klrv7z",
"drivers" : 2
},
var selector = {};
selector['vehicles.' + vehicle_number + '.drivers'] = 1;
then you use it in your query:
families.update({'_id':family_id}, {'$inc':selector} , function(err, doc) {
console.log(doc);
});
This should work.

Node.js and MongoDB Use results from one query in another

I have created a node.js module that can already query MongoDB for a set of documents using a find and output those results to JSON. My question is, knowing that node.js is asynchronous, how can I use the results from this query (items) to create a query that goes back to MongoDB to find another set of documents. This query basically returns a list of employee ids that can be used to query documents containing information on those employees(i.e. firstName, lastName etc.). Then output those results instead as JSON. The first query is basically saying, give me all of the employees that can be viewed by a particular user. I then need to take the employee ids and do a query on another set of documents that contains those individuals information, like you see below.
Here are the two documents schema:
Employee
{
"_id" : ObjectId("5208db78ecc00915e0900699"),
"clientId" : 1,
"employeeId" : "12345",
"lastName" : "DOE",
"firstName" : "JOHN",
"middleName" : "A",
"badge" : "8675309",
"birthDate" : "10/12/1978"
}
Users an employee can access (User Cache)
{
"_id" : ObjectId("520920a99bc417b7c5e36abf"),
"clientSystem" : "SystemX",
"customerNumber" : "1",
"clientUserId" : "jdoe3",
"securityCode" : "authorize",
"employeeId" : "12345",
"creationDate" : "2013-Aug-12 13:51:37"
}
Here is my code:
exports.employeeList = function(req, res) {
console.log(req.params);
var clientSystem = req.query["clientSystem"];
var clientUserId = req.query["clientUserId"];
var customerNumber = req.query["customerNumber"];
var securityCode = req.query["securityCode"];
if (clientSystem != null && clientUserId != null && customerNumber != null && securityCode != null){
db.collection('ExtEmployeeList', function(err, collection){
collection.find({'clientSystem': clientSystem, 'clientUserId':clientUserId, 'customerNumber':customerNumber, 'securityCode': securityCode}).toArray(function (err, items){
console.log(items);
res.jsonp(items);
});//close find
});//close collection
}//close if
else {
res.send(400);
}//close else
};//close function
What you're wanting to do is possible, but probably not the most effective use of Mongo. I tend to design Mongo documents around how the data will actually be used. So if I needed the user's names to show up in a list of users I can view, I would embed that data so I don't have to do multiple round trips to mongo to get all the information I need. I would do something like the following:
{
"_id" : ObjectId("520920a99bc417b7c5e36abf"),
"clientSystem" : "SystemX",
"customerNumber" : "1",
"clientUserId" : "jdoe3",
"securityCode" : "authorize",
"employeeId" : "12345",
"creationDate" : "2013-Aug-12 13:51:37"
"employee": {
"_id" : ObjectId("5208db78ecc00915e0900699"),
"clientId" : 1,
"employeeId" : "12345",
"lastName" : "DOE",
"firstName" : "JOHN",
"middleName" : "A",
"badge" : "8675309",
"birthDate" : "10/12/1978"
}
}
Yes, you are duplicating data but you're dramatically reducing the number of round trips to the database. This is typically the tradeoff you make when using document based databases since you can't join tables.

Resources