Writing Migrations with Foreign Keys Using SequelizeJS - node.js

The Background
I'm building a project with SequelizeJS, a popular ORM for NodeJS. When designing a schema, there appears to be two tactics:
Create model code and use the .sync() function to automatically generate tables for your models.
Create model code and write manual migrations using QueryInterface and umzug.
My understanding is that #1 is better for rapid prototyping, but that #2 is a best practice for projects that are expected to evolve over time and where production data needs to be able to survive migrations.
This question pertains to tactic #2.
The Question(s)
My tables have relationships which must be reflected through foreign keys.
How do I create tables with foreign key relationships with one another through the Sequelize QueryInterface?
What columns and helper tables are required by Sequelize? For example, it appears that specific columns such as createdAt or updatedAt are expected.

How do I create tables with foreign key relationships with one another through the Sequelize QueryInterface?
The .createTable() method takes in a dictionary of columns. You can see the list of valid attributes in the documentation for .define(), specifically by looking at the [attributes.column.*] rows within the params table.
To create an attribute with a foreign key relationship, use the "references" and "referencesKey" fields:
For example, the following would create a users table, and a user_emails table which references the users table.
queryInterface.createTable('users', {
id: {
type: Sequelize.INTEGER,
primaryKey: true,
autoIncrement: true
}
}).then(function() {
queryInterface.createTable('user_emails', {
userId: {
type: Sequelize.INTEGER,
references: { model: 'users', key: 'id' }
}
})
});
What columns and helper tables are required by sequelize? For example, it appears that specific columns such as createdAt or updatedAt are expected.
It appears that a standard model will expect an id, updatedAt, and createdAt column for each table.
queryInterface.createTable('users', {
id: {
type: Sequelize.INTEGER,
primaryKey: true,
autoIncrement: true
},
createdAt: {
type: Sequelize.DATE
},
updatedAt: {
type: Sequelize.DATE
}
}
If you set paranoid to true on your model, you also need a deletedAt timestamp.

I want to offer another more manual alternative because when using manual migrations and queryInterface I ran across the following problem: I had 2 files in the migration folder like so
migrations/create-project.js
migrations/create-projectType.js
because project had column projectTypeId it referenced projectType, which wasnt created yet due to the order of the files and this was causing an error.
I solved it by adding a foreign key constraint after creating both tables. In my case I decided to write it inside create-projectType.js:
queryInterface.createTable('project_type', {
// table attributes ...
})
.then(() => queryInterface.addConstraint('project', ['projectTypeId'], {
type: 'FOREIGN KEY',
name: 'FK_projectType_project', // useful if using queryInterface.removeConstraint
references: {
table: 'project_type',
field: 'id',
},
onDelete: 'no action',
onUpdate: 'no action',
}))

This is to create migration file for adding a column.
Here I want to add a column area_id in users table. Run command:
sequelize migration:create --name add-area_id-in-users
Once it gets executed creates a migration file timestamp-add-region_id-in-users in the migrations folder.
In the created migration file paste the below code:
'use strict';
module.exports = {
up: (queryInterface, Sequelize) => {
return Promise.all([
queryInterface.addColumn('users', 'region_id',
{
type: Sequelize.UUID,
references: {
model: 'regions',
key: 'id',
},
onUpdate: 'CASCADE',
onDelete: 'SET NULL',
defaultValue: null, after: 'can_maintain_system'
}),
]);
},
down: (queryInterface, Sequelize) => {
return Promise.all([
queryInterface.removeColumn('users', 'region_id'),
]);
}
};
Here in the users table I am going to create a column named region_id along with type and relation/foreign key/references. That's it.

So first one goes as the below solution and the second question: you need not explicitly mention createdAt and updatedAt because they are generated by the Sequelize for you.
The solution is:
queryInterface.createTable('Images', {
//...
}).then(
return queryInterface.addConstraint('Images', ['postId'], {
type: 'foreign key',
name: 'custom_fkey_images',
references: { //Required field
table: 'Posts',
field: 'id'
},
onDelete: 'cascade',
onUpdate: 'cascade'
})
)

Related

Create one to many relationship object in database using Sequelize

I've searched a lot and didn't find any answer.
My english isn't that good so i might not used the best keywords...
Here is my problem, is there a way to insert in one query an object like this ?
I have a one to many relationship between two objects define like this :
foo can have many bar
bar can have one foo
export const foo = sequelize.define('foos', {
idfoo: {
type: Sequelize.INTEGER,
autoIncrement: true,
primaryKey: true
},
value: Sequelize.STRING,
valueInt: Sequelize.INTEGER
})
export const bar = sequelize.define('bars', {
idbar: {
type: Sequelize.INTEGER,
autoIncrement: true,
primaryKey: true
},
value: Sequelize.STRING,
valueInt: Sequelize.INTEGER
})
foo.hasMany(bar, {
foreignKey: 'idfoo'
});
bar.belongsTo(foo);
Imagine you got this object you want to insert in your db. How would you do it ?
{
"value": "test",
"valueInt": 5,
"bars": [
{
"value": "test",
"valueInt": 6
},
{
"value": "test",
"valueInt": 7
}
]
}
I have read Sequelize documentation i didn't find any clue of this issue (or maybe i've missed it ).
I wasn't be able to make this solution works Sequelize : One-to-Many relationship not working when inserting values
Actually here is my Sequelize create function, i thought Sequelize handled all the imports alone (pretty naive i guess)...
static async post(body) {
await foo.create(body);
}
First, you need to indicate the same foreignKey option for both paired associations:
foo.hasMany(bar, {
foreignKey: 'idfoo'
});
bar.belongsTo(foo, {
foreignKey: 'idfoo'
});
That way Sequelize can make sure these two models linked by the same fields from both ends.
Second, in order to insert both main and child records together you need to indicate the include option in the create call with bar model:
static async post(body) {
await foo.create(body, {
include: [bar]
});
}

Sequelize models - Setting associations when syncing in Node

I am trying to set my associations correctly so that they pick up the foreign key values when I make GET requests in Postman. For some reason when I try to sync them there is a new column created at the end in Pgadmin4 instead of matching to the primary column value at the start.
I know Sequelize is meant to pick out the correct fields to match up but It seems to always pick up the wrong ones for the tables that are being matched up. Is there a way I could be more specific when they are initialised in my server file? I have previously used PHPmyAdmin and not had any bother but this was without an ORM.
I know about eager loading from the Sequelize docs and this is what I have managed to put together from them. Do I always need to have the associations in my server file and the model? or can I have just one? I have built my tables, had my primary data uploaded by initialising values in the server file correctly however I could never get the foreign key fields to load and the include statements never returned in my requests.
Is there a way for me to specify the column name on both tables so that the association is set-up correctly on the initial sync? At the minute I have a problem were there are columns being created with generated names of the joined up foreign keys names.
Should I be creating an index column in each of my tables that this extra column is created to match the other tables to? I don't know how to auto increment each entry to match the primary key though (which doesn't allow foreign keys when I try to set them in pgAdmin).
Finally got it sorted. The best method for this is to number every "as" key with different numbers so when you are making a findAll include call you can identify the problem straight away. I may have done a bit extra from what I am supposed to but it produces the extra field data that relates to my foreign key values.
exports.adminUserList = (req, res) => {
console.log("usegfu")
users.findAll({
include: [{
model: diveLog, as: 'userID_FK1',
}, {
model: userRole, as: 'userRoleID_FK1',
}, {
model: article, as: 'userID_FK3',
}, {
model: posts, as: 'userID_FK7',
}, {
model: photos, as: 'userID_FK5',
}],
})
.then((users) => {
server.js - association to match table models
diveLog.belongsTo(userLogin, { foreignKey: "userID", as: 'userID_FK', constraints: true, onDelete: 'cascade'});
userLogin.hasMany(diveLog, { foreignKey: "userID", as: 'userID_FK1',constraints: true, onDelete: 'cascade'});
article.belongsTo(userLogin, { foreignKey: "userID", as: 'userID_FK2',constraints: true, onDelete: 'cascade'});
userLogin.hasMany(article, { foreignKey: "userID", as: 'userID_FK3',constraints: true, onDelete: 'cascade'});
photos.belongsTo(userLogin, { foreignKey: "userID", as: 'userID_FK4',constraints: true, onDelete: 'cascade'})
userLogin.hasMany(photos, { foreignKey: "userID", as: 'userID_FK5',constraints: true, onDelete: 'cascade'});
posts.belongsTo(userLogin, { foreignKey: "userID", as: 'userID_FK6',constraints: true, onDelete: 'cascade'});
userLogin.hasMany(posts, { foreignKey: "userID", as: 'userID_FK7',constraints: true, onDelete: 'cascade'});

Understanding Sequelize database migrations and seeds

I'm trying to wrap my head around Sequelize's migrations and how they work together with seeds (or maybe migrations and seeds in general).
I set up everything to get the migrations working.
First, lets create a users table:
// migrations/01-create-users.js
module.exports = {
up: (queryInterface, Sequelize) => {
return queryInterface.createTable("Users", {
id: {
allowNull: false,
autoIncrement: true,
primaryKey: true,
type: Sequelize.INTEGER
},
email: {
type: Sequelize.STRING
},
createdAt: {
allowNull: false,
type: Sequelize.DATE,
defaultValue: Sequelize.literal('CURRENT_TIMESTAMP')
},
updatedAt: {
type: Sequelize.DATE
}
});
},
down: (queryInterface, Sequelize) => {
return queryInterface.dropTable("Users");
}
};
Fine. If I want to seed an (admin) user, I can do this as follows:
// seeders/01-demo-user.js
module.exports = {
up: (queryInterface, Sequelize) => {
return queryInterface.bulkInsert(
"Users",
[
{
email: "demo#demo.com"
}
],
{}
);
},
down: (queryInterface, Sequelize) => {
return queryInterface.bulkDelete("Users", null, {});
}
};
Then to make the magic happen, I do:
$ sequelize db:migrate
Which creates the users table in the database. After running the migrations, seeding is the next step, so:
$ sequelize db:seed:all
Tataa, now I have a user in the users database. Great.
But now I want to add firstname to the users table, so I have to add another migration:
// migrations/02-alter-users.js
module.exports = {
up: (queryInterface, Sequelize) => {
return queryInterface.addColumn("Users", "firstname", {
type: Sequelize.STRING
});
},
down: (queryInterface, Sequelize) => {
return queryInterface.removeColumn("Users", "firstname");
}
};
Running migrations again would only run the second one because it was saved in the database that the first one was already executed. But by default sequelize re-runs all seeders. So should I adjust the seeders/01-demo-user.js or change the default behavior and also store the seeders in the DB and create a new one that just updates the firstname?
What if firstname couldn't be null, then running migrations first and then the old version of seeders/01-demo-user.js would throw an error because firstname can't be null.
Re-running seeders leads to another problem: there is already a user with the demo#demo.com email. Running it a second time would duplicate the user. Or do I have to check for things like this in the seeder?
Previously, I just added the user-account in the migration so I could be sure when it was added to the DB and when I had to update it. But someone told me I was doing it all wrong and that I have to use seeders for tasks like this.
Any help/insights much appreciated.
In my opinion, a seeder is something, that is intended to run only once, while the migration is something that you add layer by layer to your DB structure continuously.
I would use seeders for populating some lookups or other data that most likely is not going to change, or test data. In the sequelize docs it's said, that "Seed files are some change in data that can be used to populate database table with sample data or test data."
If you want to make some dynamic data updates when data structure has already changed, you can run raw queries directly in your migrations if needed. So, if you, for instance, added some column in the up method, you can update the rows in the DB according to your business logic, e.g.:
// Add column, allow null at first
await queryInterface.addColumn("users", "user_type", {
type: Sequelize.STRING,
allowNull: true
});
// Update data
await queryInterface.sequelize.query("UPDATE users SET user_type = 'simple_user' WHERE is_deleted = 0;");
// Change column, disallow null
await queryInterface.changeColumn("users", "user_type", {
type: Sequelize.STRING,
allowNull: false
});
There is also an interesting discussion on this topic in Google Groups. I hope, this helps you.
In my experience migrations change structure. Seeders... seed data. Recently I was on a project that didn't have seeders configured. https://sequelize.org/master/manual/migrations.html#seed-storage. This will allow you to setup a file so your data isn't seeded more than once. Migration configuration is right there as well.
Hi try with bulkUpdate...
'use strict';
module.exports = {
up: (queryInterface, Sequelize) => queryInterface.addColumn(
'Users',
'user_type',
{
type: Sequelize.STRING,
allowNull: false,
}
).then(()=>
queryInterface.bulkUpdate('Users', {
user_type: 'simple_user',
}, {
is_deleted : 0,
})
),
down: (queryInterface, Sequelize) =>
queryInterface.removeColumn('Users', 'user_type')
};

Sequelize Nested Association with Two Tables

I have a scenario where I am trying to query a parent table (document) with two associated tables (reference & user) that do not have a relationship to each other, but do have a relationship with the parent table. In SQL, this query would look like such and correctly outputs the data I am looking for:
select *
from `document`
left join `user`
on `document`.`user_id` = `user`.`user_id`
left join `reference`
on `document`.`reference_id` = `reference`.`reference_id`
where `user`.`organization_id` = 1;
However, associations that are nested have to relate in hierarchical order in order for the query to work. Since the nested associations are not related to each other I get an association error. How can I avoid this error? Would required: false have any influence on this?
models.Document.findAll({
order: 'documentDate DESC',
include: [{
model: models.User,
where: { organizationId: req.user.organizationId },
include: [{
model: models.Reference,
}]
}],
})
Error:
Unhandled rejection Error: reference is not associated to user!
Associations:
Document:
associate: function(db) {
Document.belongsTo(db.User, {foreignKey: 'user_id'}),
Document.belongsTo(db.Reference, { foreignKey: 'reference_id'});;
}
Reference:
associate: function(db){
Reference.hasMany(db.Document, { foreignKey: 'reference_id' });
}
Should I just chain queries instead?
If you want to replicate your query (as closely as possibly) use the following query. Keep in mind that the where on the User include will only serve to remove matches on Document.user_id where the User.organization_id does not match, but the Document will still be returned. If you want to omit Documents where the User.organization_id does not match use required: true.
User <- Document -> Reference
models.Document.findAll({
// this is an array of includes, don't nest them
include: [{
model: models.User,
where: { organization_id: req.user.organization_id }, // <-- underscored HERE
required: true, // <-- JOIN to only return Documents where there is a matching User
},
{
model: models.Reference,
required: false, // <-- LEFT JOIN, return rows even if there is no match
}],
order: [['document_date', 'DESC']], // <-- underscored HERE, plus use correct format
});
The error is indicating that the User model is not associated to the Reference model, but there are only definitions for the Document and Reference models in your description. You are joining these tables in your query with the include option, so you have to make sure they are associated. You don't technically need the foreignKey here either, you are specifying the default values.
Add Reference->User association
associate: function(db) {
// belongsTo()? maybe another relationship depending on your data model
Reference.belongsTo(db.User, {foreignKey: 'user_id'});
Reference.hasMany(db.Document, { foreignKey: 'reference_id' });
}
It also looks like you probably set underscored: true in your model definitions, so your query should reflect this. Additionally if you want to perform a LEFT JOIN you need to specify required: false on the include, otherwise it is a regular JOIN and you will only get back rows with matches in the included model. You are also using the wrong order format, it should be an array of values, and to sort by model.document_date DESC you should use order: [['document_date', 'DESC']].
Proper query arguments
models.Document.findAll({
order: [['document_date', 'DESC']], // <-- underscored HERE, plus use correct format
include: [{
model: models.User,
where: { organization_id: req.user.organization_id }, // <-- underscored HERE
required: false, // <-- LEFT JOIN
include: [{
model: models.Reference,
required: false, // <-- LEFT JOIN
}]
}],
});
If you are still having trouble, try enabling logging by setting logging: console.log in your Sequelize connection, that will show you all the queries it is running in your console.
It seems to me that your problem might be the associations, trying to link back to your primary key on Documents instead of the columns 'user_id' and 'reference_id'. You didn't post the table attributes so I might have understood this wrong.
Association on documents are ok.
Documents
associate: function(db) {
Document.belongsTo(db.User, {foreignKey: 'user_id'}), //key in documents
Document.belongsTo(db.Reference, { foreignKey: 'reference_id'}); //key in documents
}
User
associate: function(db) {
User.belongsTo(db.Document, {
foreignKey: 'id', //Key in User
targetKey: 'user_id' //Key in Documents
}),
}
Reference
associate: function(db) {
Reference.belongsTo(db.Document, {
foreignKey: 'id', //Key in reference
targetKey: 'reference_id' //Key in Documents
}),
}
Also
For debbuging consider using logging so you can see the queries.
var sequelize = new Sequelize('database', 'username', 'password', {
logging: console.log
logging: function (str) {
// do your own logging
}
});

Create Association in Sequelize to do a Left Join

I want to do a left join using Sequelize that does the same as this SQL code.
SELECT * FROM events LEFT JOIN zones ON event_tz=zid WHERE event_creator_fk=116;
I have two tables: events and zones (with a list of time zones).
When querying for all the events that are created by a specific user, I also want to get the name of the time zone and other details about the TZ.
I have tried many combinations of solutions by reviewing sample code, other Stack Overflow questions and the documentation as best I can. The query always works, but doesn't do any joins. That is, it below code always returns the list of events, but no time zone data from the zones table. The generated SQL is correct, except it doesn't have the ...LEFT JOIN zones ON event_tz=zid... part.
The below code is wrong. See answers for details.
db.Event.findAll(
{ where: { event_creator_fk: someUserID } },
{ include: [{ model: db.Zone } ]}
);
If I understand correctly, adding associations between tables in sequelize results in an additional column from automatically being created. This is not what I want to do. I do not want Sequelize to modify the tables or database in any way. I want to setup my database and it's tables without Sequelize. Therefore, I am not calling sequelize.sync(). I don't know if there is away to setup associations the way I want.
Model Definitions
module.exports = function (sequelize, DataTypes) {
var Event = sequelize.define('Event', {
eid: {
type: DataTypes.INTEGER,
primaryKey: true
},
event_tz: {
type: DataTypes.INTEGER,
primaryKey: true,
references: "Zone",
referencesKey: "zid"
},
}, {
classMethods: {
associate: function (models) {
return models.Event.hasOne(models.Zone);
}
},
freezeTableName: true,
timestamps: false,
tableName: 'events'
}
);
return Event;
};
module.exports = function (sequelize, DataTypes) {
return sequelize.define('Zone', {
zid: {
type: DataTypes.INTEGER,
primaryKey: true
}
}, {
classMethods: {
associate: function (models) {
return models.Zone.belongsTo(models.Event);
}
},
freezeTableName: true,
timestamps: false,
tableName: 'zones'
});
};
Table Definitions
DROP TABLE IF EXISTS zones;
CREATE TABLE IF NOT EXISTS zones (
zid integer NOT NULL,
country_code character(2) NOT NULL,
zone_name text NOT NULL,
PRIMARY KEY (zid)
);
DROP TABLE IF EXISTS events;
CREATE TABLE IF NOT EXISTS events (
eid BIGSERIAL NOT NULL,
event_tz SERIAL NOT NULL,
PRIMARY KEY (eid),
FOREIGN KEY (event_tz)
REFERENCES zones(zid) MATCH FULL ON DELETE RESTRICT
);
You need to reverse the associations and tell sequelize about your foreign key. belongsTo means 'add the fk to this model':
models.Event.belongsTo(models.Zone, { foreignKey: 'event_tz');
models.Zone.hasOne(models.Event, { foreignKey: 'event_tz');
// or hasMany if this is 1:m
Part of the problem was that I was using the findAll method incorrectly. The query options where and include should have been included as part of the same object. The first parameter to findAll is an options parameter. See here for more details. The correct code should look like the following.
db.Event.findAll(
{
where: { event_creator_fk: someUserID },
include: [{ model: db.Zone } ]
},
);

Resources