Am i supposed to keep Sequelize models and migrations in sync? - node.js

I'm new to Sequelize.js and Databases in general, i haven't used migrations before, but i know that they can be used to make changes to the tables structure in a non-destructive way.
However i'm not sure where to declare column options (notNull, references, validate, ENUM values, etc...)
Should i declare such options in the the model file, or migration file? or both?
Wouldn't adding the options to both model and migration cause duplicate code?
(keep in mind that i'm talking about the initial migrations that create tables to the database, not the migrations that add columns and stuff...)
Any help would be appreciated!

I see three options you can take. The first two options might be edge cases but it helps to understand.
Destructive option
You want to prototype a project and you don't mind losing your data, then you could potentially do not care about migration files and synchronize your database according to your model with:
await sequelize.sync({ force: true });
It will execute on all your models:
DROP TABLE IF EXISTS "your_model" CASCADE;
CREATE TABLE IF NOT EXISTS "your_model" (...)
This command can be executed at the start of your application for example.
 Static option
As you mentioned you don't want to add columns and stuff, it's probably a good option.
Now if you don't want to lose data, you could simply use the sync method without the force option:
await sequelize.sync({ });
It will only generate:
CREATE TABLE IF NOT EXISTS "your_model" (...)
Hence, your tables are created according to your models and you don't have to create migration files.
However, if you want to modify your model and it's the most frequent usecase, the new column will not be generated in the table dynamically that's why you need migration scripts.
Flexible option
You will have to define both migration file and your model. That's what the cli does. Here is an example:
# npx sequelize-cli init or create migrations and models folders
npx sequelize-cli model:generate --name User --attributes firstName:string,email:string
Now you will have two more files:
// migrations/<date>-create-user.js
module.exports = {
up: (queryInterface, Sequelize) => {
return queryInterface.createTable('Users', {
id: {
allowNull: false,
autoIncrement: true,
primaryKey: true,
type: Sequelize.INTEGER
},
firstName: {
type: Sequelize.STRING
},
email: {
type: Sequelize.STRING
},
createdAt: {
allowNull: false,
type: Sequelize.DATE
},
updatedAt: {
allowNull: false,
type: Sequelize.DATE
}
});
},
down: (queryInterface, Sequelize) => {
// I usually remove this and create the table only if not exists
return queryInterface.dropTable('Users');
}
};
// models/users.js
module.exports = (sequelize, DataTypes) => {
const User = sequelize.define('User', {
firstName: DataTypes.STRING,
email: DataTypes.STRING
}, {});
User.associate = function(models) {
// associations can be defined here
};
return User;
};
You could refactor the code from the migration and the model, however it will be rather cumbersome because some migration files will only add one column, so merging all of them into the model might probably be less clear.

You should do it in both because as time goes by your models and inital migration will differ from each other. So I suppose you should determine a final structure in models and after that create an initial migration.

Constraints are defined and run on SQL level while validations are run on application level. Sequelize supports having validations and constraints on Models, only constraints can be defined in migrations.
My opinion is to put all constraints in migrations, and validations in Models. That way you have some kind of separation of concern as validations are run before query is made to the database - where constraints are run. You can read more on Sequelize's Validations and Constraints Validations and Constraints

Related

Create a new relationship between two tables in an already deployed database - sequelize

I'm not very familiar with sequelize, but currently I'm working with Node.js and Sequelize and I need to create a new association one to many between two tables. I know the code to generate the association:
school.hasMany(student,{ foreignKey: 'school_id', as : 'studentSchool', sourceKey: 'school_id'});
student.belongsTo(school, {foreignKey: 'school_id', targetKey : 'school_id', as: 'studentSchool'});
My problem is that the application has been deployed and in use for at least 2 years now. So there is a lot of data already. I don't know how to introduce this new association without corrupting the current data or without having to reconstruct the database.
You will need to create a migration for that.
I am assuming you already use sequelize-cli (if you do not, install from npm)
In your terminal, run
npx sequelize-cli migration:generate --name added-association-to-school-and-student
This creates an empty migration file. Fill the file with the code below
module.exports = {
up: (queryInterface, Sequelize) => {
return queryInterface.addColumn("students", "school_id", {
type: Sequelize.DataTypes.INTEGER,
/*
The defaultValue below was assigned because by default constraints are set to true.
This means that all students must belong to a school
If no school_id is specified, mysql sees that this does not follow the constraints and will opt to delete all records from the database.
So assign a default value and after this, you can go ahead to manually assign the correct schools.
ENSURE THAT THE DEFAULT VALUE OF school_id PROVIDED HAS A CORRESPONDING EXISITING RECORD IN THE school TABLE
*/
defaultValue: 1, // or any other existing value. (This is very important!)
references: {
model: "schools",
key: "school_id",
},
});
},
down: (queryInterface, Sequelize) => {
return queryInterface.removeColumn("students", "school_id");
},
};
After creating the migration file, head over to your table definitions and add the associations for the respective tables
In student Table, add this association
school.hasMany(student,{ foreignKey: 'school_id', as : 'studentSchool', sourceKey: 'school_id'});
In the school Table, add this association
student.belongsTo(school, {foreignKey: 'school_id', targetKey : 'school_id', as: 'studentSchool'});
When this is done, run the migration file in your terminal
npx sequelize-cli db:migrate
ALSO, BACK THE DATA UP BEFORE DOING THIS (Just in case)

Customizing sequelize-cli generated IDs

Created a model using:
sequelize-cli model:create --name User --attributes "dispName:string,email:string,phoneNum1:string,vendorId:integer"
Which resulted in the following migration:
'use strict';
module.exports = {
up: (queryInterface, Sequelize) => {
return queryInterface.createTable('Users', {
id: {
allowNull: false,
autoIncrement: true,
primaryKey: true,
type: Sequelize.INTEGER
},
dispName: {
type: Sequelize.STRING
},
// plus others...
});
},
down: (queryInterface, Sequelize) => {
return queryInterface.dropTable('Users');
}
};
I want to change the automatically defined ID to:
cognitoId: {
allowNull: false,
primaryKey: true,
type: Sequelize.STRING(100)
}
So:
Will sequelize be able to recognize this as the ID?
Where all do I need to make this change? I could only identify the migration file.
The model file doesn't have a definition for the cognitoId (or the original auto-generated id field): how will I be able to get the value of a User instance's cognitoId (in the data returned by queries)?
Will changing the auto-generated id field have repercussions down the line?
Is the field name id "magical"? I.e., does the primary key have to be named id?
Is there a better way to do this?
Will changing the types of the fields from Sequelize.STRING to Sequelize.STRING(100) create any issues down the line?
Why doesn't the models file generated by sequelize-cli have the id field defined?
When generating models+migrations from the command-line I couldn't find any syntax to specify the ID or any other customization for the fields.
Using:
[Node: 12.14.1, CLI: 5.5.1, ORM: 5.21.3]
PS: relatively new to NodeJS & completely new to Sequelize.
Yes
You should declare custom named PK in your model
see p.2. If you don't declare PK in your model then sequelize assumes you have id PK with an integer type, autoincremented. If you wish to assign your PK another name you should declare it in the model.
Depends on what changes you make
It is the default PK name in sequelize (see p.3). You can set different name to your PK manually declaring it in your model (see p.3)
Personally I prefer to declare all PKs in my models even if they have id name and default PK type and value.
No issues if all PK values do not exceed this length
see p.3
You can define names and types only for fields while generating models from the command line.

Why does Sequelize add extra columns to SELECT query?

When i want to get some records with joined data from the referenced tables, Sequelize adds the reference columns twice: the normal one and a copy of them, written just a little bit different.
This is my model:
module.exports = function(sequelize, DataTypes) {
return sequelize.define('result', {
id: {
type: DataTypes.INTEGER(10),
allowNull: false,
primaryKey: true,
autoIncrement: true
},
test_id: {
type: DataTypes.INTEGER(10),
allowNull: false,
references: {
model: 'test',
key: 'id'
}
},
item_id: {
type: DataTypes.INTEGER(10),
allowNull: false,
references: {
model: 'item',
key: 'id'
}
},
}, // and many other fields
{
tableName: 'result',
timestamps: false, // disable the automatic adding of createdAt and updatedAt columns
underscored:true
});
}
In my repository I have a method, which gets the result with joined data. And I defined the following associations:
const Result = connection.import('../../models/storage/result');
const Item = connection.import('../../models/storage/item');
const Test = connection.import('../../models/storage/test');
Result.belongsTo(Test, {foreignKey: 'test_id'});
Test.hasOne(Result);
Result.belongsTo(Item, {foreignKey: 'item_id'});
Item.hasOne(Result);
// Defining includes for JOIN querys
var include = [{
model: Item,
attributes: ['id', 'header_en']
}, {
model: Test,
attributes: ['label']
}];
var getResult = function(id) {
return new Promise((resolve, reject) => { // pass result
Result.findOne({
where: { id : id },
include: include,
// attributes: ['id',
// 'test_id',
// 'item_id',
// 'result',
// 'validation'
// ]
}).then(result => {
resolve(result);
});
});
}
The function produces the following query:
SELECT `result`.`id`, `result`.`test_id`, `result`.`item_id`, `result`.`result`, `result`.`validation`, `result`.`testId`, `result`.`itemId`, `item`.`id` AS `item.id`, `item`.`title` AS `item.title`, `test`.`id` AS `test.id`, `test`.`label` AS `test.label` FROM `result` AS `result` LEFT OUTER JOIN `item` AS `item` ON `result`.`item_id` = `item`.`id` LEFT OUTER JOIN `test` AS `test` ON `result`.`test_id` = `test`.`id` WHERE `result`.`id` = '1';
Notice the extra itemId, testId it wants to select from the result table. I don't know where this happens. This produces:
Unhandled rejection SequelizeDatabaseError: Unknown column 'result.testId' in 'field list'
It only works when i specify which attributes to select.
EDIT: my tables in the database already have references to other tables with item_id and test_id. Is it then unnecessary to add the associations again in the application code like I do?
A result always has one item and test it belongs to.
How can i solve this?
Thanks in advance,
Mike
SOLUTION:
Result.belongsTo(Test, {foreignKey: 'test_id'});
// Test.hasMany(Result);
Result.belongsTo(Item, {foreignKey: 'item_id'});
// Item.hasOne(Result);
Commenting out the hasOne, hasMany lines did solve the problem. I think I messed it up by defining the association twice. :|
Sequelize uses these column name by adding an id to the model name by default. If you want to stop it, there is an option that you need to specify.
underscored: true
You can specify this property on application level and on model level.
Also, you can turn off the timestamps as well. You need to use the timestamp option.
timestamps: false
Although your solution fixes your immediate problem, it is ultimately not what you should be doing, as the cause of your problem is misunderstood there. For example, you MUST make that sort of association if making a Super Many-to-Many relationship (which was my problem that I was trying to solve when I found this thread). Fortunately, the Sequelize documentation addresses this under Aliases and custom key names.
Sequelize automatically aliases the foreign key unless you tell it specifically what to use, so test_id becomes testId, and item_id becomes itemId by default. Since those fields are not defined in your Result table, Sequelize assumes they exist when generating the insert set, and fails when the receiving table turns out not to have them! So your issue is less associating tables twice than it is that one association is assuming extra, non-existing fields.
I suspect a more complete solution for your issue would be the following:
Solution
Result.belongsTo(Test, {foreignKey: 'test_id'});
Test.hasMany(Result, {foreignKey: 'test_id'});
Result.belongsTo(Item, {foreignKey: 'item_id'});
Item.hasOne(Result, {foreignKey: 'item_id'});
A similar solution fixed my nearly identical problem with some M:N tables.

Sequelize how not to redefine models every time server starts

In Sequelize tutorials, it is said that a single model is generated in this way:
const User = sequelize.define('user', {
firstName: {
type: Sequelize.STRING
},
lastName: {
type: Sequelize.STRING
}
});
And than saved (i.e. create table) like this :
User.sync().then(() => {
// do whatever
});
But I expect to do that just once, I need to create tables just once. So the next time I run the script how to just retrieve models (i.e. tables) that were defined before with the above code.
in sync method you can pass an option to avoid sync of database tables every time. This option will make sure that your application checks for table in database, if it exist then it will not create otherwise it will create table.
User.sync(
{force: false}).then(() => {
// do whatever
});
Let me know if you still face issue. I am using sequalize and i am not getting this issue.

Creating Static Values into a Sequelize model (no experience with migration)

I am working on a group project that tests our use of Sequelize vs. normal ORM generated back-end items. Two of our models in a MySQL DB are category tables. These will not be dynamically created, updated or destroyed, but need to be there when the program runs. The class I am a part of hasn't covered instances or migrations. Here is the model thus far..
module.exports = function(sequelize, DataTypes){
var maincategories = sequelize.define("maincategories", {
maincategories_id: {
//make primary key
type: DataTypes.INTEGER,
allowNull: false,
primaryKey: true,
autoIncrement:true
},
maincategories_name: {
type: DataTypes.STRING,
allowNull: false
}
},
{
classMethods: {
associate: function(models){
maincategories.hasMany(models.posts),
maincategories.hasMany(models.subcategories)
}
}
});
// maincategories.create({ maincategories_name: 'For Sale'}).then(function(insertedCategory){
// console.log(insertedCategory.dataValues);
// });
maincategories.create({ maincategories_name: 'Housing'})
maincategories.create({ maincategories_name: 'Personals'});
return maincategories;
// maincategories.hasMany(posts);
// maincategories.hasMany(subcategories);
};
How can I get the Category table to have those values added to it at the time or prior to the node.js app starting? Also, would the code reside in the model, api route or a separate file that is required in somewhere else? As you can see I tried to do persistent instances but these did not work. It would state that I had created these items in node CLI but nothing showed in the actual database.
Thank you.
You need maincategories.sync() in order for sequelize to sync the defined models to your db.
http://docs.sequelizejs.com/en/v3/api/sequelize/#syncoptions-promise

Resources