I'm trying to deep populate a collection.
For example
// UnitType.js
name: { type: 'string' }
// Unit.js
unitType: {
model: 'unitType',
via: '_id',
required: true,
index: true
}
// Product.js
unit: {
model: 'unit',
via: '_id',
required: true,
index: true
},
The problem is, that - as far I know from internet research deep populate like
Product.find().populate('unit.unitType');
is currently not supported in sails. To achieve the result I want I currently
query Products with populate unit
query UnitTypes with the id from `product.unit.unitType``
.map() product.unit.unitType with the response
This is of course far from ideal. I also tried using toJSON in the model to "pre-populate" the unitType -> doesn't work since this doesn't support Promises.
There are quite a few threads on so and PR's on github on this issue, but so far I haven't found a solution to this problem. Is there any way to make this better?
You could try to replace Waterline ORM with Offshore ORM. There is a hook for sails to do that - sails-hook-orm-offshore.
It's quite easy to implement to your existing project, because its fork of Waterline wits some more features. Only cons i found is that sails-hook-validation
stopped working.
How to install
npm install sails-hook-orm-offshore --save
Configuration
.sailsrc
{
"hooks": {
"orm": false,
"pubsub": false
}
}
Defaults
config/globals.js
adapters: true,
models: true
Usage
Now you will be allowed to deep populate in your queries. For example (from documentation):
User.find()
.populate('foo.bar', { name: 'foo' }) //populate foo into user and bar into foo
.exec(function(err, users) {
console.log(users[0].foo.bar.name) // 'foo'
});
Second option
Merge deep populate with waterline
npm i Atlantis-Software/waterline#deepPopulate
Related
We are running:
NodeJS v13.11
Sequelize v5.22.3
Postgres v11.7-2
I have 3 models, GameVersion, Tag, and TagTaggable (representing the associative entity). A GV can have many Tags, and a Tag can be associated with many GVs (or any other model via TagTaggable).
Models are built with this (not showing the hooks to handle and clean-up after the polymorphism):
GameVersion:
this.belongsToMany(models.Tag, {
through: {
model: models.TagTaggable,
unique: false,
scope: {
taggableType: 'game_version',
},
},
as: 'tags',
foreignKey: 'taggable_id',
constraints: false,
});
this.hasMany(models.TagTaggable, {
foreignKey: 'taggable_id',
scope: {
taggableType: 'game_version',
},
});
Tag:
this.belongsToMany(models.GameVersion, {
through: {
model: models.TagTaggable,
unique: false,
scope: {
taggableType: 'game_version',
},
},
as: 'gameVersions',
foreignKey: 'tag_id',
constraints: false,
});
this.hasMany(models.TagTaggable, {
scope: {
taggableType: 'game_version',
},
});
TagTaggable:
this.belongsTo(models.Tag, {
as: 'tag',
foreignKey: 'tag_id',
});
// Polymorphic relationships
this.belongsTo(models.GameVersion, {
foreignKey: 'taggable_id',
constraints: false,
as: 'gameVersion',
});
The Tags are applied to a GV by:
await gv.setTags(metadata.tags);
where metadata.tags is a collection of Tag models.
This works perfectly, the first time I run it, and I can see that, debugging into the bowels of the Sequelize BelongsToMany.updataAssociations method, the scope is correct on that first time, by looking at this.through.scope on the BelongsToMany object.
I get:
{taggableType: 'game_version'}
Notice the camel-cased key, which is what it should be.
This results in the following query:
INSERT INTO "tag_taggable" ("taggable_type","created_at","updated_at","taggable_id","tag_id") VALUES ('game_version','2020-11-27 18:49:34.767 +00:00','2020-11-27 18:49:34.767 +00:00',82,29)
The problem arises on any subsequent attempt (meaning it works the first time after starting the server, but any attempts prior to re-starting the server fail), and I can see that this.through.scope now results in:
{taggable_type: 'game_version'}
Notice the snake-cased key.
This results in the following query (notice the lack of the "taggable_type" column):
INSERT INTO "tag_taggable" ("created_at","updated_at","taggable_id","tag_id") VALUES ('2020-11-27 18:51:16.423 +00:00','2020-11-27 18:51:16.423 +00:00',83,29)
and throws a "not-null constraint violation."
This SEEMS to be down in the guts of Sequelize, but I cannot imagine it not being surfaced already if it were (unless polymorphic m:n is really uncommon).
Has anyone had this experience, and/or can anyone shed any light on what is going on here?
I would really like to use the setTags magic method, but I am at the point of just hand-building the TagTaggable objects and stuffing them in the DB myself.
TIA for any insights/assistance,
Chris
I am beginner in sailsjs, i have multiple databases but table structure is same. So i would like to use single model and controller for all these databases.
I know we can set connection attribute in model like this
module.exports = {
connection: 'mysqlServer1',
attributes: {
userid: {
type: 'integer',
primaryKey: true
},
fbid: {
type: 'string'
},
source: {
type: 'string'
}
}
}
};
But how can we set connection dynamically runtime?
Thanks
I know this is old, but I stumbled on it looking for a solution to different problem, thought I'd answer.
The simplest way is to just set environment variables and use defaults.
For example, if you put MODELA_CONN="mysqlServer1" in your .bash_profile, or lift sails with an export like export MODELA_CONN="mysqlServer1" && sails lift, then you can just use that:
module.exports = {
connection: process.env.MODELA_CONN || "defaultMysql",
...
}
I am afraid this isn't possible yet. The closest thing you can get to a dynamic connection is using the following npm package https://github.com/sgress454/sails-hook-autoreload.
This will automatically reload sailjs with the changed connection config, without to lift sails again. But it won't cover your problem during runtime.
Working on a project in KeystoneJS and I'm having trouble figuring out the mongoose relationship bit.
According to the keystone docs, let's say we have the following models: User and Post. Now a post has a relationship to a user, so I'll write:
Post.add({
author: { type: Types.Relationship, ref: 'User' }
});
and then:
User.relationship({ path: 'posts', ref: 'Post', refPath: 'author' });
Now, I want to be able to see all posts related to that User without having to query for both a User and Posts. For example, if I queried for a user object I would like to be able to do user.posts and have access to those related posts. Can you do this with mongoose/keystone?
As far as I understand, keystone's List Relationship has nothing to do with mongoose and querying. Instead, it is used by keystone's admin UI to build out the relationship queries before rendering them in the view. This said I would forget User.relationship(...); solving your problem, although you want it for what I just mentioned.
The following should work fine based on your schema, but only populates the relationship on the one side:
var keystone = require('keystone');
keystone.list('Post').model.findOne().populate('author', function (err, doc) {
console.log(doc.author.name); // Joe
console.log(doc.populated('author')); // '1234af9876b024c680d111a1' -> _id
});
You could also try this the other way, however...
keystone.list('User').model.findOne().populate('posts', function (err, doc) {
doc.posts.forEach(function (post) {
console.log(post);
});
});
...mongoose expects that this definition is added to the Schema. This relationship is added by including this line in your User list file:
User.schema.add({ posts: { type: Types.Relationship, ref: 'Post', many: true } })
After reading the keystone docs, this seems to be logically equivalent the mongoose pure way, User.schema.add({ posts: [{ type: Schema.Types.ObjectId, ref: 'Post' }] });. And now you are now maintaining the relationship on both lists. Instead, you may want to add a method to your keystone list.
User.schema.methods.posts = function(done){
return keystone.list('Post').model.find()
.where('author', this.id )
.exec(done);
};
By adding a method to your User list, it saves you from persisting the array of ObjectIds relating the MongoDB document back to the Post documents. I know this requires a second query, but one of these two options look to be your best bet.
I found this on their github https://github.com/Automattic/mongoose/issues/1888, check it for context, but basically says to use the keystone populateRelated() method. I tested it and does work
// if you've got a single document you want to populate a relationship on, it's neater
Category.model.findOne().exec(function(err, category) {
category.populateRelated('posts', function(err) {
// posts is populated
});
});
I'm aware the question is old but this has to be out there for further reference
I'm playing with model associations in sails and I'm curious if it's possible to make a query base on the associated field.
Example:
User.js
attributes:{
classes: { collection: 'Class', via: 'students' }
}
Class.js
attributes: {
type: ...
students: { collection: 'User', via: 'classes'}
}
Is there a way to retrieve specific Classes of a Student base on the type of class because right now everything is being returned when I use .populate(). (maybe similar with the logic below)
User
.findOne({name: 'StudentA'})
.populate('classes')
.where({'classes.type':['type1', 'type2']})
.then(....)
Thanks
You can add a where clause to your populate like so:
User
.findOne({name: 'StudentA'})
.populate('classes', {where: {type: ['type1', 'type2']}})
.exec(...)
In addition to where, you can also use skip, limit and sort in the second argument to populate.
Keep in mind this is still (as of this posting) in beta, so if you find any issues where it seems to not be working correctly, please post them to the Waterline GitHub issues forum.
We're rapidly developing an application that's using Mongoose, and our schema's are changing often. I can't seem to figure out the proper way to update a schema for existing documents, without blowing them away and completely re-recreating them from scratch.
I came across http://mongoosejs.com/docs/api.html#schema_Schema-add, which looks to be right. There's little to no documentation on how to actually implement this, making it very hard for someone who is new to MongoDB.
I simply want to add a new field called enabled. My schema definition is:
var sweepstakesSchema = new Schema({
client_id: {
type: Schema.Types.ObjectId,
ref: 'Client',
index: true
},
name: {
type: String,
default: 'Sweepstakes',
},
design: {
images: {
type: [],
default: []
},
elements: {
type: [],
default: []
}
},
enabled: {
type: Boolean,
default: false
},
schedule: {
start: {
type: Date,
default: Date.now
},
end: {
type: Date,
default: Date.now
}
},
submissions: {
type: Number,
default: 0
}
});
Considering your Mongoose model name as sweepstakesModel,
this code would add enabled field with boolean value false to all the pre-existing documents in your collection:
db.sweepstakesModel.find( { enabled : { $exists : false } } ).forEach(
function (doc) {
doc.enabled = false;
db.sweepstakesModel.save(doc);
}
)
There's nothing built into Mongoose regarding migrating existing documents to comply with a schema change. You need to do that in your own code, as needed. In a case like the new enabled field, it's probably cleanest to write your code so that it treats a missing enabled field as if it was set to false so you don't have to touch the existing docs.
As far as the schema change itself, you just update your Schema definition as you've shown, but changes like new fields with default values will only affect new documents going forward.
I was also searching for something like migrations, but didn't find it. As an alternative you could use defaults. If a key has a default and the key doesn't exist, it will use the default.
Mongoose Defaults
Default values are applied when the document skeleton is constructed. This means that if you create a new document (new MyModel) or if you find an existing document (MyModel.findById), both will have defaults provided that a certain key is missing.
I had the exact same issue, and found that using findOneAndUpdate() rather than calling save allowed us to update the schema file, without having to delete all the old documents first.
I can post a code snippet if requested.
You might use mongo shell to update the existing documents in a specific collection
db.SweeptakesModel.update({}, {$set: {"enabled": false}}, {upsert:false, multi:true})
I had a similar requirement of having to add to an existing schema when building an app with Node, and only found this (long ago posted) query to help.
The schema I added to by introducing the line in the original description of the schema and then running something similar to the following line, just the once, to update existing records:
myModelObject.updateMany( { enabled : { $exists : false } }, { enabled : false } )
'updateMany' being the function I wanted to mention here.
just addition to what Vickar was suggesting, here Mongoose Example written on Javascript (Nodejs):
const mongoose = require('mongoose');
const SweeptakesModel = mongoose.model(Constants.SWEEPTAKES,sweepstakesSchema);
SweeptakesModel.find( { enabled : { $exists : false } }).then(
function(doc){
doc.enabled = false;
doc.save();
}
)