Aggregate Data into Array MongoDB, Mongoose, Node - node.js

Ok, here is what I am trying to do. I have a user schema which has a team key and an array as its value. For example:
user = {
team : Array
}
The values of the array are strings (team names). I have a team schema that has the team information, such as roster, state, etc. Here is my question. If my user has more than one team name in the array, how then would I aggregate all the data to one variable and then pass it to my view. I thought I could just use a for loop, but the view seems to render before the aggregation is done. Here is what I tried:
// if there is there is more than one team name in the user's team array
if( data[0].teams.length > 1 ){
for(var i = 0; i < data[0].teams.length; i++){
// for each team name in the user array
// find the corresponding team and add that data to the array called team
Team.find( { team : data[0].teams[i] }, function(err, data){
if(err){
throw err;
}
team.push(data);
});
}
// render the view
res.render('coach', {
user: req.user,
fname: self.fname,
lname: self.lname,
email: self.email,
phone: self.phone,
address: self.address,
state: self.state,
teams : [team]
});
}
Edit (added schemas)
User schema ( look in teams array, find corresponding teams, return data from team schema based on the name in the user schema teams array )
var userSchema = new Schema({
username : { type: String, required: true, index: { unique: true } },
fname : { type: String, required: true },
lname : { type: String, required: true },
password : { type: String, required: true },
type : { type: String, required: true },
phone : String,
address : String,
state: String,
conference: String,
email : { type: String, required: true, index: { unique : true } },
teams: Array,
});
Team Schema:
var teamsSchema = new Schema({
coach: String,
state: String,
conference: String,
team: String,
roster: [{fname: String, lname: String, number: String}]
});

Now that you have posted your schema It is clear what you are trying to do and also clear how to show your various approaches to do this much better.
The first and basic case is that you seem to have an array of "teams" containing the "String" value for the "team name" presumably stored on your user object. This appears to work for you and does have the advantage of having those names accessible as you retrieve the user.
The thing is that you are iterating over the results and issuing a .find() for every element in the array, which is not the most efficient way. You can basically use the $in operator with your query and existing data and get around what you probably tried to do in merging the full "team data" to the user object by calling .toObject() to transform the mongoose document to an plain JavaScript object:
User.findById( req.user, function(err,user) {
var data = user.toObject();
Team.find({ "team": "$in": data.teams }, function(err,teams) {
data.teams = teams;
res.render( 'coach', data );
});
});
That is the simple approach, but really there are features available to mongoose that will sort of do this for you if you change your schema a little to reference the data in the Team model. So you can then just fill in using .populate():
var userSchema = new Schema({
username : { type: String, required: true, index: { unique: true } },
fname : { type: String, required: true },
lname : { type: String, required: true },
password : { type: String, required: true },
type : { type: String, required: true },
phone : String,
address : String,
state: String,
conference: String,
email : { type: String, required: true, index: { unique : true } },
teams: [{ type: ObjectId, ref: "Team" }]
});
Now that stores just the references _id value and where to get it from. So now your query becomes this:
User.findById( req.user).populate("teams").exec(function(err,user) {
res.render("coach", user);
});
So what this does is basically the equivalent of the first example but using the _id values and in a little cleaner way, and the user object now has pulled in all of the data from the Team schema matching the list of _id values stored in your user array.
Of course you don't have the same immediate access to just the team name as you did before, but you can work around this by selecting the fields to populate with something like this:
User.findById( req.user).populate("teams","team").exec(function(err,user) {
res.render("coach", user);
});
So that would only pull in the "team" field from each object and is much the same as what you originally had as a result, albeit with actually two queries (under the hood).
Finally if you can live with the concept of a little data replication, then the most efficient way is to simply embed the data. While there is duplicated data being stored,the reading is the most efficient as it is a single query. If you don't need to regularly update that data, this may be worth consideration:
var userSchema = new Schema({
username : { type: String, required: true, index: { unique: true } },
fname : { type: String, required: true },
lname : { type: String, required: true },
password : { type: String, required: true },
type : { type: String, required: true },
phone : String,
address : String,
state: String,
conference: String,
email : { type: String, required: true, index: { unique : true } },
teams: [teamSchema]
});
So with that data embedded you only need to retrieve the user and the "teams" data would already be there:
User.findById( req.user, function(err,user) {
res.render("coach", user);
});
So those are a few approaches to take. All show that the looping of the array is not required, and that the number of queries you are actually issuing can be greatly reduced and even down to 1 if you can live with that.

Firstly, the find operations are asynchronous and although you are waiting for a response before pushing to the team array you are not waiting for all responses before responding with res.render(). Something like below should show what I mean:
// if there is there is more than one team name in the user's team array
if( data[0].teams ){
var teamArr = [];
var responses = 0;
for(var i=0; i<data[0].teams.length; i++){
// for each team name in the user array
// find the corresponding team and add that data to the array called team
Team.find( { team : data[0].teams[i] }, function(err, team){
if(err){
throw err;
}
responses ++;
teamArr.push(team);
// if all teams have been pushed then call render
if (responses == data[0].teams.length - 1) {
// render the view
res.render('coach', {
user: req.user,
fname: self.fname,
lname: self.lname,
email: self.email,
phone: self.phone,
address: self.address,
state: self.state,
teams : teamArr
});
}
});
}
}
Edit: I changed some naming to avoid potential naming clashes (two data variables for example)
Secondly, another option if you change your document structure, given you are using mongoose, if you stored references to the team document objects in your array instead of name strings you could use the mongoose populate method and populate the array before returning the whole document.
Update: Based on Schema provided you could use populate as follows:
var userSchema = new Schema({
username : { type: String, required: true, index: { unique: true } },
....
email : { type: String, required: true, index: { unique : true } },
teams: [{
type: Schema.Types.ObjectId, ref: 'teams'
}],
});
And then populate the document before returning it avoiding the for loop:
User.findOne({_id: userId})
.populate('teams')
.exec(function (err, user) {
if (err) return handleError(err);
res.render('coach', {
user: filterUser(user);
});
});
where user.teams now contains a populated array of teams and filterUser() returns a user object with only the properties you require;

Related

Insert a document with mongoose without initialize the model with empty attributes

I want to insert a document in my database from a website form. I have a model created with mongoose and I want to save in the database only the attributes that contains data and I don't want to save empty attributes.
This is my model:
const localizationSchema = new Schema({
name: { type: String, required: true },
spins: [{ type: String }],
spinsForChild: [{ type: String }],
parent: { id: String, name: String },
localizationType: { type: String },
count: { type: Number, default: 0 },
countries: [{ id: String, name: String, cities: [{ id: String, name: String }] }]
});
const Localization = mongoose.model('Localization', localizationSchema);
When I try to save a new document, it creates in the database all attributes although I don't send it on my query.
Localization.create({
name: body.name,
localizationType: body.localizationType,
"parent.id": parent.id,
"parent.name": parent.name,
spins: spins,
spinsForChild: spinsForChild
}, function(err) {
if (err) return handleError(err);
res.redirect('/localizations');
});
This code, for example, inserts in the DB an empty array called "countries".
I tried to use strict: false in the model declaration but it didn't works.
You could use this answer.
But, thing you try to implement seems to be anti-pattern and can cause errors when you will try to use array update operators with undefined array. So, use it carefully.
Good luck!

Mongo user document structure with three user types

I'm setting up a Mongo database in Express with Mongoose and I'm trying to decide how to model the users. I've never modeled multiple users in the MEAN stack before and thought I'd reach out for some best-practices - I'm an instructor and need to be able to teach my students best practices. I haven't been able to find a whole lot out there, but perhaps I'm searching for the wrong things.
The app will have 3 user types, student, staff, and admin. Each user type will require some of the same basics - email, password, first and last names, phone, etc. If the user is a student, they will need to provide additional info like their high school name, grade, age, gender, etc, which ideally will be required.
This is what I've come up with so far - a single user model that requires all the basic information, but also has schema set up to allow for the additional information that students will need to include. Then I also have a pre-save hook set up to remove the "studentInfo" subdocument if the user being saved doesn't have a "student" role:
var mongoose = require("mongoose");
var Schema = mongoose.Schema;
var ethnicityList = [
"White",
"Hispanic or Latino",
"Black or African American",
"Native American or American Indian",
"Asian / Pacific Islander",
"Other"
];
var userSchema = new Schema({
firstName: {
type: String,
required: true
},
lastName: {
type: String,
required: true
},
phone: {
type: Number,
required: true
},
email: {
type: String,
required: true,
lowercase: true,
unique: true
},
password: {
type: String,
required: true
},
preferredLocation: {
type: String,
enum: ["provo", "slc", "ogden"]
},
role: {
type: String,
enum: ["student", "staff", "admin"],
required: true
},
studentInfo: {
school: String,
currentGrade: Number,
ethnicity: {
type: String,
enum: ethnicityList
},
gender: {
type: String,
enum: ["male", "female"]
}
}
}, {timestamps: true});
userSchema.pre("save", function (next) {
var user = this;
if (Object.keys(user.studentInfo).length === 0 && user.role !== "student") {
delete user.studentInfo;
next();
}
next();
});
module.exports = mongoose.model("User", userSchema);
Question 1: Is this an okay way to do this, or would it be better just to create two different models and keep them totally separate?
Question 2: If I am going to be to restrict access to users by their user type, this will be easy to check by the user's role property with the above setup. But if it's better to go with separated models/collections for different user types, how do I check whether its a "Staff" or "Student" who is trying to access a protected resource?
Question 3: It seems like if I do the setup as outlined above, I can't do certain validation on the subdocument - I want to require students to fill out the information in the subdocument, but not staff or admin users. When I set any of the fields to required, it throws an error when they're not included, even though the subdocument itself isn't required. (Which makes sense, but I'm not sure how to get around. Maybe custom validation pre-save as well? I've never written that before so I'm not sure how, but I can look that up if that's the best way.)
Well, Here are my two cents.
You would be better off creating separate schema models and then injecting the models on a need to basis.
for e.g.
If I have a blog schema as follows:
var createdDate = require('../plugins/createdDate');
// define the schema
var schema = mongoose.Schema({
title: { type: String, trim: true }
, body: String
, author: { type: String, ref: 'User' }
})
// add created date property
schema.plugin(createdDate);
Notice that author is referring to User and there is an additional field createdData
And here is the User Schema:
var mongoose = require('mongoose');
var createdDate = require('../plugins/createdDate');
var validEmail = require('../helpers/validate/email');
var schema = mongoose.Schema({
_id: { type: String, lowercase: true, trim: true,validate: validEmail }
, name: { first: String, last: String }
, salt: { type: String, required: true }
, hash: { type: String, required: true }
, created: {type:Date, default: Date.now}
});
// add created date property
schema.plugin(createdDate);
// properties that do not get saved to the db
schema.virtual('fullname').get(function () {
return this.name.first + ' ' + this.name.last;
})
module.exports = mongoose.model('User', schema);
And the created Property which is being refereed in both User and Blogspot
// add a "created" property to our documents
module.exports = function (schema) {
schema.add({ created: { type: Date, default: Date.now }})
}
If you want to restrict access based on the user types, you would have to write custom validation like in the User schema we had written for emails:
var validator = require('email-validator');
module.exports = function (email) {
return validator.validate(email);
}
And then add an if-else based on whatever validations you do.
2 and 3. So, Yes custom validations pre-save as well.
Since you are an instructor I preferred to just point out the practices that are used instead of elaborating on your specific problem.
Hope this helps! :)

Mongoose: how to structure a schema with a SET of subdocuments (one unique field)?

I'm trying to make the following schema to work:
var FormSchema = new mongoose.Schema({
form_code: { type: String, unique: true },
...
});
var UserSchema = new mongoose.Schema({
...
submissions: [{
form_code: { type: String, unique: true },
last_update: Date,
questions: [{
question_code: String,
answers: [Number]
}]
}],
});
The rationale here is that a user can have many unique forms submitted, but only the last submission of each unique form should be saved. So, ideally, by pushing a submission subdocument when updating a user, the schema would either add the submission object to the set, or update the subdocument containing that form_code.
The following code doesn't work as desired (it pushes the new subdocument even if the form_code is already present):
User.findOneAndUpdate(
{ _id: user.id },
{ $addToSet: { submissions: submission_object } },
function (err, user) {
// will eventually have duplicates of form_code at user.submissions
}
);
The above schema clearly doesn't work, what must be changed to achieve that "upsertToSet"?

Mongoose Populate - array

can someone please help me with population of this schema? I need to populate array of Staff by their userId.
var PlaceSchema = new Schema ({
name: { type: String, required: true, trim: true },
permalink: { type: String },
country: { type: String, required: true },
...long story :D...
staff: [staffSchema],
admins: [adminSchema],
masterPlace:{ type: Boolean },
images: []
});
var staffSchema = new Schema ({
userId: { type: Schema.Types.ObjectId, ref: 'Account' },
role: { type: Number }
});
var adminSchema = new Schema ({
userId: { type: Schema.Types.ObjectId, ref: 'Account'}
})
var Places = mongoose.model('Places', PlaceSchema);
I tried to use this query, but without success.
Places.findOne({'_id' : placeId}).populate('staff.userId').exec(function(err, doc){
console.log(doc);
});
Polpulation is intended as a method for "pulling in" information from the related models in the collection. So rather than specifying a related field "directly", instead reference the related fields so the document appears to have all of those sub-documents embedded in the response:
Places.findOne({'_id' : placeId}).populate('staff','_id')
.exec(function(err, doc){
console.log(doc);
});
The second argument just returns the field that you want. So it "filters" the response.
There is more information on populate in the documentation.

Mongoose Relationship Populate Doesn't Return results

var SecuritySchema = new Mongoose.Schema({
_bids: [{
type: Mongoose.Schema.Types.ObjectId,
ref: 'BuyOrder'
}],
_asks: [{
type: Mongoose.Schema.Types.ObjectId,
ref: 'SellOrder'
}]
});
var OrdersSchema = new Mongoose.Schema({
_security: {
type: Mongoose.Schema.Types.ObjectId,
ref: 'Security'
},
price: {
type: Number,
required: true
},
quantity: {
type: Number,
required: true
}
});
// declare seat covers here too
var models = {
Security: Mongoose.model('Security', SecuritySchema),
BuyOrder: Mongoose.model('BuyOrder', OrdersSchema),
SellOrder: Mongoose.model('SellOrder', OrdersSchema)
};
return models;
And than when I save a new BuyOrder for example:
// I put the 'id' of the security: order.__security = security._id on the client-side
var order = new models.BuyOrder(req.body.order);
order.save(function(err) {
if (err) return console.log(err);
});
And attempt to re-retrieve the associated security:
models.Security.findById(req.params.id).populate({
path: '_bids'
}).exec(function(err, security) {
// the '_bids' array is empty.
});
I think this is some sort of naming issue, but I'm not sure, I've seen examples here and on the moongoose website that use Number as the Id type: http://mongoosejs.com/docs/populate.html
The ref field should use the singular model name
Also, just do:
models.Security.findById(req.params.id).populate('_bids').exec(...
My main suspicion given your snippet at the moment is your req.body.order has _security as a string instead of an array containing a string.
Also, you don't need an id property. Mongodb itself will automatically do the _id as a real BSON ObjectId, and mongoose will add id as a string representation of the same value, so don't worry about that.
While I don't understand your schema (and the circular nature of it?), this code works:
var order = new models.BuyOrder({ price: 100, quantity: 5});
order.save(function(err, orderDoc) {
var security = new models.Security();
security._bids.push(orderDoc);
security.save(function(err, doc) {
models.Security.findById({ _id: doc._id })
.populate("_bids").exec(function(err, security) {
console.log(security);
});
});
});
It:
creates a BuyOrder
saves it
creates a Security
adds to the array of _bids the new orderDoc's _id
saves it
searches for the match and populates
Note that there's not an automatic method for adding the document to the array of _bids, so I've done that manually.
Results:
{ _id: 5224e73af7c90a2017000002,
__v: 0,
_asks: [],
_bids: [ { price: 100,
quantity: 5,
_id: 5224e72ef7c90a2017000001, __v: 0 } ] }

Resources