I am using Sequlizejs 2.0.0-rc4.
I have two entities: Feeds (containing an owner_id field) and Owners. Basically, there is a one-to-many relationship between Feeds and Owners - an Owner has many Feeds.
I have tried modelling this many different ways, but the only way I got it right is:
Owner.hasMany(Feed, {foreignKey: 'owner_id'});
Feed.belongsTo(Owner, {foreignKey: 'owner_id'});
The only problem with this is the fact that when I try to get a Feed like so:
models.feeds.findAll({
include: [models.owners]
}).then(function (feeds) {
// feeds will contain a SequlizeInstance object
});
Then the result will contain a SequlizeInstance object instead of a Owner object, which I then have to access through something like feeds.owners.dataValues.
What is the problem here?
There is no problem here :). Sequelize returns sequelize instances because they are what allows you to do all the lovely sequelize stuff, such as feed.owner.updateAttributes({.. }).
If you want the plain values from the owner object, you should call get or toJSON
Related
For example, if I had a 'Conversation' model a simple chat messaging system, I might do the following:
module.exports = {
attributes: {
messages: {
collection: 'Message'
}
}
}
Is this allowed in SailsJs? If not, is it recommended to mimic a "Has" relationship from Conversation to Message by using some form of custom array? Such as below:
module.exports = {
attributes: {
messages: {
type: 'array'
}
}
}
In a more complex scenario, my goal is to have the 'Conversation' know all of its 'Message' objects, but it is unnecessary for those 'Message' objects to know of its associated 'Conversation'.
I'd been using that construct for quite a while but only now did I find that the official docs don't specify it.
They mention that in one-way associations a model is associated with another model and don't mention collections. (Though they should work in just the same manner.)
For one-to-many associations they specify that a model can be associated with many other models (a collection) but don't specify what happens if you ignore the via attribute. They simply mention it is needed.
However, if you simply leave out the via attribute, the id field is used as the key for the association. So the construct you specified is allowed.
On a different note, you might want to reconsider keeping messages as either an array or a collection. Since you might need to add/retrieve/update/remove messages in a random fashion and collections and arrays can only be accessed as a whole, it might make sense to specify a relevant index on the Message collection and forgo having an association. This would let you quickly run queries like "retrieve the last 10 messages of thread " and so on.
My app currently uses the Sequelize sync() method to create the database, and I want to change it to use the migrations system.
One of my model has belongsTo() associations with other models, and I don't really know how to make the initial migration code for these associations.
Do I have to manually create the foreign key with SQL queries, or is there some methods available?
Case 1: Database initialization
If your purpose is to add relations during initialization of database structure it is better to just use sync method instead of manually adding them using migrations. If your models are properly designed and have relations defined, they will be created automatically during execution of sync method.
Take a look at sequelize express example. In models directory you have three files:
index.js - which includes all models
task.js - task model
user.js - user model
Look at task.js content, starting from line 7 the following code creates a relation between User and Task models:
classMethods: {
associate: function(models) {
Task.belongsTo(models.User, {
onDelete: "CASCADE",
foreignKey: {
allowNull: false
}
});
}
}
If you correctly prepare your relations in model files, sync will create the foreign keys for you. Migrations aren't necessary in this case.
I encourage you to read the whole express-example readme.md and browse repository files to see how the things work with express and sequelize.
Case 2: Database structure migration
In case you already have some data which you want to keep, you need to use migration script, because the only way for sync to restructure your database is to destroy it completely alongside with all its data.
You can read about basic migrations in the sequelize docs. Unfortunately docs do not cover creating a relation. Let's assume you want to create the following relation: User belongs to Group. To create column on the user side of relation, you may use addColumn method.
queryInterface.addColumn(
'user',
'group_id',
{
type: Sequelize.INTEGER,
allowNull: true
}
)
Unfortunately there isn't a nice function (yet) to create the foreign key constraint for you, but you can do it manually using sequelize query method. Postgresql example:
queryInterface.sequelize.query("ALTER TABLE user
ADD CONSTRAINT user_group_id_fkey FOREIGN KEY (group_id)
REFERENCES group (id) MATCH SIMPLE
ON UPDATE CASCADE ON DELETE CASCADE;");
Edit: Added database structure migration case
Adding this as an answer instead of a comment (not enough rep) for #aryeh-armon answer above. It's the table name that you need to make sure exists rather than the model name. i.e. if your model is named Job and your db table is named Jobs then the migration would look look like this instead.
jobId: {
type: Sequelize.INTEGER,
references: {
model: "Jobs",
key: "id"
}
},
you can add references to the migrations
Exmaple:
user_id: {
type: Sequelize.BIGINT,
references: {
model: "users",
key: "id"
}
},
Just make sure the model you are referencing exists.
After lots of searching, I found a couple of blog posts explaining what I wanted to do. The first one no longer exists, and here's the second one
Apparently it's not really the common way to do it, but it seems more logical to me. If you want to use only the migrations, you have to use SQL queries to create the initial migration.
But anyway, I think that ezpn is right about creating the initial database with sync, and then migrate. It seems easier than using umzug and only use migrations.
I'm using Sequelize.js for ORM and have a few associations (which actually doesn't matter now). My models get get and set methods from those associations. Like this (from docs):
var User = sequelize.define('User', {/* ... */})
var Project = sequelize.define('Project', {/* ... */})
// One-way associations
Project.hasOne(User)
/*
...
Furthermore, Project.prototype will gain the methods getUser and setUser
according to the first parameter passed to define.
*/
So now, I have Project.getUser(), which returns a Promise. But if I call this twice on the very same object, I get SQL query executed twice.
My question is - am I missing something out, or this is an expected behavior? I actually don't want to make additional queries each time I call the same method on this object.
If this is expected - should I use custom getters with member variables which I manually populate and return if present? Or there is something more clever? :)
Update
As from DeBuGGeR's answer - I understand I can use includes when making a query in order to eager load everything, but I simply don't need it, and I can't do it all the time. It's waste of resources and a big overhead if I load my entire DB at the beginning, just to understand (by some criteria) that I won't need it. I want to make additional queries depending on situation. But I also can't afford to destroy all models (DAO objects) that I have and create new ones, with all the info inside them. I should be able to update parts of them, which are missing (from relations).
If you use getUser() it will make the query call, it dosent give you access to the user. You can manually save it to project.user or project.users depending on the association.
But you can try Eager Loading
Project.find({
include: [
{ model: User, as: 'user' } // here you HAVE to specify the same alias as you did in your association
]
}).success(function(project){
project.user // contains the user
});
Also e.g of getUser(). Dont expect it to automatically cache user and dont override this cleverly as it will create side effects. getUser is expected to get from database and it should!
Project.getUser().then(function(user){
// user is available and is a sequelize object
project.user = user; // save project.user and use it till u want to
})
The first part of things is clear - every call to get[Association] (for example Project.getUser()) WILL result in database query.
Sequelize does not maintain any kind of state nor cache for the results. You can get user in the Promisified result of the call, but if you want it again - you will have to make another query.
What #DeBuGGeR said - about using accessors is also not true - accessors are present only immediately after a query, and are not preserved.
As sometimes this is not ok, you have to implement some kind of caching system by yourself. Here comes the tricky part:
IF you want to use the same get method Project.getUser(), you won't be able to do it, as Sequelize overrides your instanceMethods. For example, if you have the association mentioned above, this won't work:
instanceMethods: {
getUser: function() {
// check if you have it, otherwise make a query
}
}
There are few possible ways to fix it - either change Sequelize core a little (to first check if the method exists), or use some kind of wrapper to those functions.
More details about this can be found here: https://github.com/sequelize/sequelize/issues/3707
Thanks to mickhansen for the cooperation on how to understand what to do :)
I had a very weird issue with the way Mongoose interacted with my Node and Mongo database.
I was using express to create a basic get api route to fetch some data from my mongodb.
I had a database called test and it had a collection call "billings"
so the schema and route was pretty basic
apiRouter.route('/billing/')
.get(function(req, res) {
Billing.find(function(err, billings) {
if (err) res.send(err);
// return the bills
res.json(billings);
});
});
Where "Billing" was my mongoose schema. that simply had 1 object {test: string}
This worked fine, I got a response with all the items in my mongo db called "billings" which is only one item {test: "success"}
Next I created a collection called "historys"
I setup the exact same setup as my billings.
apiRouter.route('/historys/')
// get all the history
.get(function(req, res) {
Historys.find(function(err, historys) {
if (err) res.send(err);
// return the history
res.json(historys);
});
});
where again "Historys" was my mongoose schema. This schema was identical in setup to my billings since I didnt have any real data, the fields were the same, i just had it with a test field so the json object returned from both billings and historys should have been
{ test: "success" }
However, this time I didnt get any data back, I just got an empty object
[].
I went through my code multiple times to make sure maybe a capital got lost, or a comma somewhere etc, but the code was identical. the setup and formatting in my mongodb was identical. I went into robomongo and viewed the database and everything was named correctly.
Except, I had 2 new collections now.
My original : "Historys" AND a brand new collection "Histories"
Once i fixed my api route to go look at Histories instead of Historys, I was able to get the test data successfully. I still however cannot pull data from Historys, its like it doesnt exist yet there it was in my robomongo console when I refreshed.
I searched all my code for any mention of histories and got 0 results. Where did the system know to fix the grammar on my collection?
From the docs:
When no collection argument is passed, Mongoose produces a collection name by passing the model name to the utils.toCollectionName method. This method pluralizes the name. If you don't like this behavior, either pass a collection name or set your schemas collection name option.
So, when you did, in your schema definition, this:
mongoose.model('Historys', YourSchema);
, mongoose created the Histories collection.
When you do:
db.historys.insert({ test: "success" })
through mongodb console, if the historys collection doesn't exist, it'll be created. That's why you have the two collections in your db. Like the docs said, if you don't want mongoose to create a collection with a pluralized name based on your model, just specify the name you want.
I'm working on a custom adapter in sails#0.10.0-rc4 which will support associations but I am having trouble getting them working in conjunction with my adapter. My configuration is a one-to-many association between article and stats. My models and adapter are setup like this:
// api/models/article.js
module.exports = {
connection: ['myadapter'],
tableName: 'Knowledge_Base__kav',
attributes: {
KnowledgeArticleId: { type: 'string', primaryKey: true }
stats: {
collection: 'stats',
via: 'parentId'
}
}
// api/models/stats.js
module.exports = {
connection: ['myadapter'],
tableName: 'KnowledgeArticleViewStat',
attributes: {
count: 'integer',
ParentId: {
model: 'article'
}
}
}
// adapter.js
find: function(connectionName, collectionName, options, cb) {
console.dir(options)
// output
// {where: null}
db.query(options, function(err, res)) {
cb(err, res)
}
}
However, when I try to populate using Article.find().populate('stats').exec(console.log()), my adapter gets {where: null} as options when I would expect it to receive {where: {parentId: [<some-article-id>]}}. It will return a list of articles to me but the field which is supposed to be populated from another model (stats) is just an empty list.
I feel like this is related to the fact that my adapter is not getting the proper where param to search for the related model on the primary key. To test this further, I setup a test one-to-many relationship using the the sails-mongo adapter. In this case the adapter did receive params I expected and the association worked fine.
Does anyone have any idea on why .populate('stats') wouldn't be sending the proper "where" params to my adapter?
Update 3/7
So it seems like what happens in associations is that SomeModel.find() will hit the adapter once and then .populate('othermodel') hits the adapter again using the primary key of the first request. Then the results of both are joined together. In my case, the second hit against the adapter isn't happening for some unknown reason.
Update
The original issue was related to an attribute naming error that's mentioned in the comments below. However, there still appears to be some issue with the final population step mentioned by particlebanana:
Final step will: Take all of the query results from all the returned query operations
and combine them in-memory to build up a result set you can return in
the exec callback.
I'm seeing that all required queries are now firing but they are failing to actually populate the alias. Here's the call with some added debugging output in the form of a gist for easier consumption: https://gist.github.com/jasonsims/9423170
It looks like you are on the right track! The way the operation sets get built up, the .find() on the Article should run with the first log (empty where) and the second query should get run with the parentId criteria in the log. The second query isn't running because it can't build up that parentId array of primary keys when you don't return anything from the first query.
Short answer: you need to return something in the find callback to see the second log, which should match your expected criteria.
The query lifecycle looks something like this:
Check if all query pieces are on the same connection, if not break out which queries will run on which connections
For all queries on a single connection, check if the adapter supports native joins (has a .join() method, if so you can pass the criteria down and let the adapter handle the joins.
If no native join method is defined run the "parent" operation (in this case the Article.find())
Use the results of the parent operation to build up criteria for any populations that need to run. (The parentId array in your criteria) and run the child results.
Take all of the query results from all the returned query operations and combine them in-memory to build up a result set you can return in the exec callback.
I hope that helps some. Shoot me the url of your repo and I will look through it, if it's able to be open sourced, and can help some more if you come across any issues.
Just to summarize, there were multiple issues going on here which were causing associations not to populate:
Custom primary keys
There was a problem with waterline when joining data from models using custom primary keys. #particlebanana fixed this in 8eff54b and it should be included in the next rc of waterline (waterline#0.10.0-rc5).
Malformed SOQL query
When waterline queries the adapter for a second time in order to acquire the child rows, it does so using { foreignKey: [ value ] }. Since the value was a list, jsforce was incorrectly generating the SOQL query since it expected all list values to be accompanied by either $in or $nin operators. I addressed this issue in github/jsforce#9 and it's now included in jsforce#1.1.2.
Model attributes are case sensitive
The models in my project were defined in snakeCase but the json response from Salesforce was using EveryWordCapitalized. This causes 1-to-many joins in waterline to reduce the many child records to one when it runs _.uniq(childRows, pk). Since the model has defined pk == id but the actual value returned from Salesforce is pk == Id, this call to uniq blows away all child records but one. I'm not entirely sure if this should be a waterline bug or not but fixing the capitalization in the model attribute definitions resolved this.