mongodb: a query issue - node.js

I am designing a MEAN (MongoDb + Express.js + Angular.js + Node.js) app.
The application actors are users and persons; currently there are ~1000 persons and ~100 users.
The users are the application registered users, and the persons are external people the users need to be informed of.
Each user is able to rate and take some notes about any person she is interested in.
The database schemas I'm planning are:
var person = new mongoose.Schema({
name: String,
phone: String,
...
};
var user = new mongoose.Schema({
name: String,
email: String,
...
},
var userToPersonSchema = new mongoose.Schema({
userId: { type: ObjectId , required: true },
personId: { type: ObjectId, required: true },
rating: Number,
notes: String,
...
});
This is the query I plan to add user rating for a person:
db.userToPerson.insert({
userId: currentUserId,
personId: currentPersonId,
rating: 10,
notes: 'my preferred person!'
});
This is the code I have to find all persons with a rating by a user:
var currentUserId = '123...';
var personsAll = db.person.find();
var usersToPersonsAll = db.userToPerson.find({ _id: currentUserId });
var personsRatedByCurrentUser = [];
for (var p = 0; p < personsAll.length; p++) {
for (var u = 0; u < usersToPersonsAll.length; u++) {
if (personsAll[p]._id === usersToPersonsAll[u].personId) {
personsRatedByCurrentUser.push(personsAll[p]);
}
}
}
The question:
for the last "query" I suppose I'd better use some form of aggregation, but I can't find out any...
Any advise about a schema design modification should be welcome, too, of course...

Any time I need a join in MongoDB, I break the problem into two queries.
First, fetch the ids from the first collection using distinct. Distinct just returns an array of unique values.
Then, query the second collection for documents corresponding to those ids. The $in parameter conveniently takes an array.
var currentUserId = '123...';
var personIds = db.userToPerson.distinct("personId", { _id: currentUserId });
var personsRatedByCurrentUser = db.person.find({ _id: {$in, personIds}});

Related

How do I create a number of objects with insertMany with existing data from the db?

I'm currently trying to insert a large number of models through insertMany, but I can't seem to figure out how to populate the array when creating an object. I'm relatively new to Mongoose and any help would be appreciated, here is the code I have right now.
const ProgramsSchema = new mongoose.Schema({
program_id: {
type: String,
required: true
},
description: {
type: String
},
});
const schoolsSchema = new mongoose.Schema({
inst_url: {
type: String
},
programs: {
type: [{type: ProgramsSchema, ref: "Programs"}]
}
});
And here's the code where I try to create a number of schools and add it to the database.
let new_schools = []
for (let i = 0; i < schools.length; i++) {
let school = schools[i]
let p_arr = []
for (let p_index = 0; p_index < school["PROGRAMS"].length; p_index++) {
let p_id = school["PROGRAMS"][p_index]
Programs.find({program_id: p_id}).populate('Programs').exec(function(err, data) {
if (err) {
console.log(err);
} else {
p_arr.push(data[0])
}
})
}
let newSchool = {
inst_url: school["INSTURL"],
programs: p_arr,
}
new_schools.push(newSchool);
}
Schools.insertMany(new_schools);
I can basically add all of the school data into the db, but none of the programs are being populated. I was wondering if there was a way to do this and what the best practice was. Please let me know if you guys need more info or if my question wasn't clear.
There are a few problems with your mongoose schemas. The operation you are trying to do in find is not available, based on your mongoose schemas. You cannot populate from "Programs" to "Schools". You can populate from "Schools" to "Programs", for instance:
Schools.find().populate(programs)
And to do that, several changes in your schemas are necessary. The idea is to store the programs _id in your programs array in School collection and be able to get the programs info through populate(), either regular populate or 'custom populate' (populate virtuals).
Regular populate()
I would change the schoolsSchema in order to store an array of _id into programs:
const schoolsSchema = new mongoose.Schema({
inst_url: {
type: String
},
programs: [
{type: String, ref: "Programs"}
]
});
You should change ProgramsSchema as well:
const ProgramsSchema = new mongoose.Schema({
_id: Schema.Types.ObjectId, // that's important
description: {
type: String
},
});
And now, you can do:
Programs.find({_id: p_id}).exec(function(err, data) {
if (err) {
console.log(err);
} else {
p_arr.push(data[0]._id)
}
})
Your documents should be inserted correctly. And now you can populate programs when you are performing a query over School, as I indicated above:
Schools.find().populate(programs)
Populate Virtual
The another way. First of all, I have never tried this way, but I think it works as follows:
If you want to populate over fields that are not ObjectId, you can use populate virtuals (https://mongoosejs.com/docs/populate.html#populate-virtuals). In that case, your schemas should be:
const ProgramsSchema = new mongoose.Schema({
program_id: String,
description: {
type: String
},
});
const schoolsSchema = new mongoose.Schema({
inst_url: {
type: String
},
programs: [
{type: String, ref: "Programs"}
]
});
Enable virtual in your School schema:
Schools.virtual('programs', {
ref: 'Programs',
localField: 'programs',
foreignField: 'program_id'
});
Then, you should store the program_id.
Programs.find({program_id: p_id}).exec(function(err, data) {
if (err) {
console.log(err);
} else {
p_arr.push(data[0].program_id)
}
})
And as before, you can populate() when you need.
I hope I helped

mongoose-schema - are both children array + parent doc's ID necessary?

I'm using a couple of one-to-many models and was wondering what the advantage of having both an array of "children" ObjectID()s and a "parent" model's ObjectID() in the child is. For example:
// a client will have a new card every ten visits
var ClientSchema = new Schema({
first_name: String,
last_name: String,
email: String,
cards: [] // ObjectID()s, <--- is this necessary?
});
var CardSchema = new Schema({
client: ObjectID(), // <--- or is this enough?
visits: []
});
I think the client: ObjectID() should do the trick in most cases, specially with the Population options Mongoose offers.
It suffices to store the reference ObjectId in one of the documents.
As you can read in the documentation or in this answer, the reference field needs a type and a ref. The name of field is arbitrary. Once you have done this and registered your models, you can use the models to populate your queries.
var clientSchema = new Schema({
first_name: String,
last_name: String,
email: String
});
var cardSchema = new Schema({
client: {
type: Schema.Types.ObjectId,
ref: 'client'
}
});
// models
var Card = mongoose.model('Card', cardSchema);
var Client = mongoose.model('Client', clientSchema);
Yet, it could be helpful to store an array of ObjectId's. However, this also creates a huge window for mistakes. A mismatch of foreign key and primary key, in SQL dialect. You can always do a count query if you need to count. I would say, do either the above or do this:
var clientSchema = new Schema({
first_name: String,
last_name: String,
email: String,
cards: [
{
type: Schema.Types.ObjectId,
ref: 'card'
}
]
});
var cardSchema = new Schema({
card_name: String
});
// models
var Card = mongoose.model('Card', cardSchema);
var Client = mongoose.model('Client', clientSchema);

Mongoose: Exclude objects from results based on another model

Let's say I have a User and Group model, groups have users, like
var GroupSchema = mongoose.Schema({
name: String,
users: [{ type: mongoose.Schema.ObjectId, ref: 'User' }]
});
How would I query to get all the Users but exclude the ones that are on
Group.users, I'm already doing this by querying first Group then manually filtering against all users
var groupP = Group.findById(group_id).populate('users');
var userP = User.find();
Promise.props({
group: groupPromise.exec(),
users: usersPromise.exec()
})
.then(function (result) {
//this gives the expected result but I'm looking for a more straight forward mongoose only solution if possible
var users = differenceWith(result.users, result.group.users, (a, b) => { return a._id.toString() == b._id.toString()});
})
You can try below query.
var groupP = Group.findById(group_id);
var userP = User.find({_id:{$nin:groupP.users}});

How to find count from two collections relationship in MongoDB using Mongoose

I want to display list files of particular user (_id: 876896) with their click counts as below:
Sr. No. | File Name | Click Count
Below is the sample schema I am using:
var clicks = mongoose.Schema({
file_id : String,
IP : String
});
var files = mongoose.Schema({
filename : String,
owner : {type:mongoose.Schema.ObjectId, ref:'users'},
});
How effectively this can done.
You can do it in two step, first get all files that refer to the users you want data for. Then, get alls clicks that are related to files you read.
There is no INNER_JOIN in mongodb
Little example:
files.find({
owner: {
$in: usersIdsArray // [_id, _id...] ids of all users
},
}).then((ret = []) => {
// Here ret is array of files
if (!ret.length) // There is no files matching the user
return clicks.find({
file_id: {
$in: Array.from(ret, x => x._id.toString()),
// build an array like [_id, _id, _id...]
},
});
}).then((ret = []) => {
// Here you have all clicks items
});
I also recommend to use embedded schema instead of multiple collections:
var clicks = mongoose.Schema({
file_id: String,
IP: String
});
var files = mongoose.Schema({
filename: String,
owner: {type:mongoose.Schema.ObjectId, ref:'users'},
});
turn into:
var files = mongoose.Schema({
filename: String,
owner: {type:mongoose.Schema.ObjectId, ref:'users'},
clicks: [String], // With String your IP
});
MongoDB is good when it comes to big data, but not that good in relations.

how to implement the function like left join of mysql in mongoose

I am going to implement the function like left join of mysql in mongoose.
the date is
var mongoose = require('mongoose')
, Schema = mongoose.Schema
var personSchema = Schema({
_id : Number,
name : String
});
var storySchema = Schema({
_creator : { type: Number, ref: 'Person' },
title : String
});
var personProfile = Schema({
userid : {type: Number, ref: 'Person'},
birthday: Date,
profilelink: String,
email: String
});
var Story = mongoose.model('Story', storySchema);
var Person = mongoose.model('Person', personSchema);
var personProfile = mongoose.model('Personprofile', personProfile );
I am going to display the Story model with the user profile.
We have to get the profile info with the _creator of story and the userid of personProfile
How can I get the info using mongoose query?
Thanks Nelis
What your are trying to do is not possible because there is no join statement on mongodb.
You can achieve this in two ways:
1 - By DBRefs: Changing your Schema to one that include all the user info and do not split them in two different schemas as you are doing, see denormalized. Then you can use the Population function to get all the persons data.
2 - By Manual references: The second solution is to make a second call to the database getting the personProfile data using the userid as a filter.
Example 1:
This way you can get all persons data without a second call to the database.
var personSchema = Schema({
_id : Number,
name : String,
birthday: Date,
profilelink: String,
email: String
});
var storySchema = Schema({
_creator : { type : Schema.Types.ObjectId, ref: 'Person' },
title : String
});
Story
.find()
.populate(['_creator'])
.exec(function(err, stories) {
//do your stuff here
}
Notice that I'm using the type Schema.Types.ObjectId and not the Number. This way, you can assign a new value to _creator passing either the _id or the person object and the mongoose will convert the object to its _id. For example, you can post something like
{
_creator : {
_id : 123123123123,
name : 'Foo',
birthday: '0000-00-00',
profilelink: 'http://foo.bar',
email: 'foo#bar.com'
},
title : 'Mr'
}
... and the mongoose will convert to
{
_creator : 123123123123,
title : 'Mr'
}
Example 2:
This way your data still normalized and you can get all the persons data with a second call.
Story
.find()
.exec(function(err, stories) {
var arrayLength = stories.length;
for (var i = 0; i < arrayLength; i++) {
var story = stories[i];
personProfile.findById(story._creator, function (err, person) {
story._creator = person;
}
};
// do your stuff here
}

Resources