TL;DR: Is there a safe way to dynamically define a mongoose discriminator at runtime?
I have an app with a MongoDB collection where users have some control over the underlying schema.
I could add one or two fixed, required fields and just use mongoose.Mixed for the remainder that users can change, but I'd like to make use of Mongoose's validation and discriminators if I can.
So, what I've got is a second collection Grid where the users can define the shape they'd like their data to take, and in my main model Record, I've added a function to dynamically generate a discriminator from the definition in the second collection.
The code for my Record model looks like this:
const mongoose = require("mongoose")
const recordSchema = new mongoose.Schema({
fields: {
type: Array,
required: true
}
}, {
discriminatorKey: "grid"
})
const Record = mongoose.model("Record", recordSchema)
module.exports = grid => {
// Generate a mongoose-compatible schema from the grid's field definitions
const schema = grid.fields.map(field => {
if(field.type === "string") return { [field.name]: String }
if(field.type === "number") return { [field.name]: Number }
if(field.type === "checkbox") return { [field.name]: Boolean }
return { [field.name]: mongoose.Mixed }
})
return Record.discriminator(grid._id, new mongoose.Schema(schema))
}
This is inside an Express app, and I use the model in my middleware handlers something like this:
async (req, res) => {
const grid = await Grid.findById(req.params.id)
const Record = await GenerateRecordModel(grid)
const records = await Record.find({})
res.json({
...grid,
records
})
}
This works great on the first request, but after that I get an error Discriminator with name “ ” already exists.
I guess this is because only one discriminator with its name per model can exist.
I could give every discriminator a unique name whenever the function is called:
return Record.discriminator(uuidv4(), new mongoose.Schema(schema), grid._id)
But I imagine that this isn't a good idea because discriminators seem to persist beyond the lifetime of the request, so am I laying the groundwork for a memory leak?
I can see two ways forward:
COMPLICATED? Define all discriminators when the app boots up, rather than just when a HTTP request comes in, and write piles of extra logic to handle the user creating, updating or deleting the definitions over in the Grid collection.
SIMPLER? Abandon using discriminators, just use mongoose.Mixed so anything goes as far as mongoose is concerned, and write any validation myself.
Any ideas?
I am creating two models, the first one is User and the second one is Portfolio. When a user is creating a Portfolio (where a user can only have one Portfolio), I want it to have reference to the user who is creating it, and every time that user's data is fetched, I want it to also fetching their portfolio data if any.
I am trying to use hasOne to create portfolio_id inside User tables, with the skeleton generated using sequelize init command, but it is not working. I cannot find a column the name protfolio_id if I don't put it inside the user migration file. Is that how it is supposed to be?
How should I design the models? Should I include the portfolio_id in User tables and include user_id in Portfolio table, or is there a best way to do it?
And which associations method should I use, hasOne or belongsTo?
First of all make sure that you are calling Model.associate for each model. This will run queries for all the relationships.
You can define the relationships in the associate method as follows:
// user.js (User Model definition)
module.exports = (sequelize, dataTypes) => {
const { STRING } = dataTypes
const User = sequelize.define("user", {
username: { type: STRING }
})
User.associate = models => {
User.hasOne(models.Portfolio, { foreignKey: "userId" }) // If only one portfolio per user
User.hasMany(models.Portfolio) // if many portfolios per user
}
return User
}
// portfolio.js (Portfolio Model definition)
module.exports = (sequelize, dataTypes) => {
const { STRING } = dataTypes
const Portfolio = sequelize.define("portfolio", {
portfolioName: { type: STRING }
})
Portfolio.associate = models => {
Portfolio.belongsTo(models.User, { foreignKey: "userId" })
}
return Portfolio
}
hasOne stores the foreignKey in the target model. So this relationship will add a foreign key userId to the Portfolio model.
belongsTo stores the key in the current model and references the primary key of the target model. In this case the Portfolio.belongsTo will add userId in the Portfolio model which will reference the primary key of User model.
Notice how both these relationships do the same thing, they add userId to the Portfolio model. Its better to define this in both models for your last use case:
I want it to have reference to the user who is creating it, and every time that user's data is fetched, I want it to also fetching their portfolio data if any.
Accessing related models:
In sequelize fetching a related model together with the main model is called Eager Loading. Read more about it here.
Now for your use case if you want to fetch the portfolio if any, while fetching user, do the following:
var userWithPortfolio = await User.findAll({include: [models.Portfolio]};
// Or you may also use include: {all: true} to include all related models.
var userWithPortfolio = await User.findAll({include: {all: true}};
/*
Output:
userWithPortfolio = {
username: "xyz",
portfolio: {
portfolioName: "xyz"
}
}
*/
I have a sequelize model called Activity, that belongs to another model called User.
I want to be able to generate a dynamic property on the Activity model, so I did the following:
class Activity extends Sequelize.Model {
toJSON() {
return {
...this.dataValues,
my_property: 'its value', // Dynamic property
};
}
}
This works quite fine when I query the Activity itself (or get them all), but the property is not there when I query a User with its activities e.g.
const user = await User.findByPk('some-pk-here', { include: [Activity] });
Can this behavior be changed, and if yes then how?
While different from a previous question I asked here, it's related and so wanted to link it.
I have been trying hard to find out how I can get the model name (identity) or model "class" (exposed in sails.models) of a record. So, given a waterline record, how can I find out its model name or class?
Example (of course here I know the model is User but that is an example):
User.findOne(1).exec(function(err, record) {
// at this point think that we don't know it's a `user` record
// we just know it's some record of any kind
// and I want to make some helper so that:
getTheModelSomehow(record);
// which would return either a string 'user' or the `User` pseudo-class object
});
I have tried to access it with record.constructor but that is not User, and I couldn't find any property on record exposing either the model's pseudo-class object or the record's model name.
UPDATE:
To clarify, I want a function to which I'll give ANY record, and which would return the model of that record as either the model name or the model pseudo-class object as in sails.models namespace.
modelForRecord(record) // => 'user' (or whatever string being the name of the record's model)
or
modelForRecord(record) // => User (or whatever record's model)
WOW, ok after hours of research, here is how I am doing it for those who are interested (it's a very tricky hack, but for now unable to find another way of doing):
Let's say record is what you get from a findOne, create, ... in the callback, to find out what instance it is, and so find the name of the model owning the record, you have to loop over all your models (sails.models.*) and make an instanceof call this way:
function modelFor(record) {
var model;
for (var key in sails.models) {
model = sails.models[key];
if ( record instanceof model._model.__bindData__[0] ) {
break;
}
model = undefined;
}
return model;
}
Do not try to simply do instanceof model, that does not work
After if you need the model name simply modelFor(record).globalId to get it.
In your model definition why not just create a model attribute. Then the model will be returned with every record call. This will work even after the record become a JSON object.
module.exports = {
attributes : {
model : {type:'string',default:'User'}
}
}
Use this example for parsing model name
var parseModel = function(request) {
request = request.toLowerCase();
return sails.models[request];
};
and use in controller this code for using this model
parseModel(req.param('modelname')).find().then().catch();
Sails exposes everything in the request object. Try grabbing the name of the model this way:
var model = req.options.model || req.options.controller;
That will give you the raw name. To use it, you'll have to plug the model into the sails models array.
var Model = req._sails.models[model];
Check out the source code to see it in action. (https://github.com/balderdashy/sails/blob/master/lib/hooks/blueprints/actionUtil.js#L259)
It seems Mongoose is doing something really funky internally.
var Foo = new mongoose.model('Foo', new mongoose.Schema({a: String, b: Number}));
var foo = new Foo({a: 'test'; b: 42});
var obj = {c: 1};
foo.goo = obj; // simple object assignment. obj should be
// passed by reference to foo.goo. recall goo
// is not defined in the Foo model schema
console.log(foo.goo === obj); // comparison directly after the assignment
// => false, doesn't behave like normal JS object
Essentially, any time you try to deal with properties of a Mongoose model that aren't
a) defined in the model's schema or
b) defined as the same type (array, obj, ..) ... the model doesn't even behave like a normal Javascript object.
Switching line 4 to foo._doc.goo = obj makes the console output true.
edit: trying to reproduce weirdness
example 1:
// Customer has a property 'name', but no property 'text'
// I do this because I need to transform my data slightly before sending it
// to client.
models.Customer.find({}, function(err, data) {
for (var i=0, len=data.length; i<len; ++i) {
data[i] = data[i]._doc; // if I don't do this, returned data
// has no 'text' property
data[i].text = data[i].name;
}
res.json({success: err, response:data});
});
_doc exist on the mongoose object.
Because mongooseModel.findOne returns the model itself, the model has structure (protected fields).
When you try to print the object with console.log it gives you only the data from the database, because console.log will print the object public fields.
If you try something like JSON.stringify then you get to see inside the mongoose model object. (_doc, state ...)
In the case where you want to add more fields in the object and it's not working
const car = model.findOne({_id:'1'})
car.someNewProp = true // this will not work
If later you set the property to the object car and you didn't specify in the Model Schema before then Mongoose model is validating if this field exists and if it's the valid type.
If the validation fails then the property will not be set.
Update
Maybe I misunderstood your original question, but now it looks like the nature of your question changed, so the below information isn't relevant, but I'm leaving it. :)
I tested your code and it works fine for me. Mongoose doesn't execute any special code when you set properties that aren't part of the schema (or a few other special properties). JavaScript currently doesn't support calling code for properties that don't yet exist (so Mongoose can't get in the way of the set of the goo property for example).
So, when you set the property:
foo.goo = { c: 1 };
Mongoose isn't involved. If your console.log was something other than the code you displayed, I could see that it might report incorrectly.
Additionally, when you send the results back as JSON, JSON.stringify is being called, which calls toString on your Mongoose Model. When that happens, Mongoose only uses the properties defined on the schema. So, no additional properties are being sent back by default. You've changed the nature of the data array though to directly point at the Mongoose data, so it avoids that problem.
Details about normal behavior
When you set the property goo using Mongoose, quite a few things happen. Mongoose creates property getters/setters via the Object.defineProperty (some docs). So, when you set the goo property, which you've defined as a [String], a few things happen:
Mongoose code is called prior to the value being set onto the object instance (unlike a simple JavaScript object)
Mongoose creates an array (optionally) to store the data (a MongooseArray) which will contain the array data. In the example you provided, since you didn't pass an array, it will be created.
Mongoose will attempt to cast your data to the right type
It will call toString on the data passed as part of the cast.
So, the results are that the document now contains an array with a toString version of the object you passed.
If you checked the contents of the goo property, you'd see that it's now an array with a single element, which is a string that contains [object Object]. If you'd picked a more basic type or matched the destination property storage type, you would see that a basic equality check would have worked.
you can use toJSON() instead of _doc
Try using lean
By default, Mongoose queries return an instance of the Mongoose Document class. Documents are much heavier than vanilla JavaScript objects, because they have a lot of internal state for change tracking. Enabling the lean option tells Mongoose to skip instantiating a full Mongoose document and just give you the POJO.
https://mongoosejs.com/docs/tutorials/lean.html
Had same problem. Instead of updating my model.
const car = model.findOne({_id:'1'})
let temp = JSON.stringify(car);
let objCar = JSON.parse(temp);
objCar.color = 'Red'; //now add any property you want
this solves my problem
I was stuck on this today... Drove me nuts. Not sure if the below is a good solution (and OP has mentioned it too), but this is how I overcame this issue.
My car object:
cars = [{"make" : "Toyota"}, {"make" : "Kia"}];
Action:
console.log("1. Cars before the color: " + car);
cars.forEach(function(car){
car.colour = "Black"; //color is NOT defined in the model.
});
console.log("2. Cars after the color: " + car);
Problematic console output:
1. Cars before the color: [{"make" : "Toyota"}, {"make" : "Kia"}];
2. Cars after the color: [{"make" : "Toyota"}, {"make" : "Kia"}]; //No change! No new colour properties :(
If you try to pass in this property that was undefined in the model, via doc (e.g. car._doc.color = "black"), it will work (this colour property will be assigned to each car), but you can't seem to access it via EJS (frontend) for some reason.
Solution:
(Again, not sure if this is the best way... but it worked for me): Add in this new property (colour) in the car model.
var carSchema = mongoose.Schema({
make: String,
color: String //New property.
})
With the model redefined, everything worked as normal / expected (no _doc 'hacks' needed etc.) and I lived another day; hope it helps someone else.
There is some weirdness with Mongoose models and you have to check that Mongoose doesn't already have a model created in it's models array.
Here is my solution:
import mongoose from 'mongoose';
createModel = (modelName="foo", schemaDef, schemaOptions = {})=> {
const { Schema } = mongoose;
const schema = Schema(schemaDef, schemaOptions);
const Model = mongoose.models[modelName] || mongoose.model(modelName, schema);
return Model;
}
I use my own mongoose model class and base class for my models. I made this and it should work for you.
For those using spread(...) and/ can't see a solution, here's an example of #entesar's answer
Instead of spread or ._doc in:
import User from "./models/user";
...
async function createUser(req, res) {
const user = await User.create(req.body);
res.status(201).json({
message: "user created",
data: {
...user // OR user._doc,
token: "xxxxxxxx",
},
});
}
...
Use this
import User from "./models/user";
...
async function createUser(req, res) {
const user = await User.create(req.body);
res.status(201).json({
message: "user created",
data: {
...user.toJSON(),
token: "xxxxxxxx",
},
});
}
...
Ps: took me a while to understand the answer.
You should add .lean() on the find to have it skip all the Model "magic".