Understanding Sequelize database migrations and seeds - node.js

I'm trying to wrap my head around Sequelize's migrations and how they work together with seeds (or maybe migrations and seeds in general).
I set up everything to get the migrations working.
First, lets create a users table:
// migrations/01-create-users.js
module.exports = {
up: (queryInterface, Sequelize) => {
return queryInterface.createTable("Users", {
id: {
allowNull: false,
autoIncrement: true,
primaryKey: true,
type: Sequelize.INTEGER
},
email: {
type: Sequelize.STRING
},
createdAt: {
allowNull: false,
type: Sequelize.DATE,
defaultValue: Sequelize.literal('CURRENT_TIMESTAMP')
},
updatedAt: {
type: Sequelize.DATE
}
});
},
down: (queryInterface, Sequelize) => {
return queryInterface.dropTable("Users");
}
};
Fine. If I want to seed an (admin) user, I can do this as follows:
// seeders/01-demo-user.js
module.exports = {
up: (queryInterface, Sequelize) => {
return queryInterface.bulkInsert(
"Users",
[
{
email: "demo#demo.com"
}
],
{}
);
},
down: (queryInterface, Sequelize) => {
return queryInterface.bulkDelete("Users", null, {});
}
};
Then to make the magic happen, I do:
$ sequelize db:migrate
Which creates the users table in the database. After running the migrations, seeding is the next step, so:
$ sequelize db:seed:all
Tataa, now I have a user in the users database. Great.
But now I want to add firstname to the users table, so I have to add another migration:
// migrations/02-alter-users.js
module.exports = {
up: (queryInterface, Sequelize) => {
return queryInterface.addColumn("Users", "firstname", {
type: Sequelize.STRING
});
},
down: (queryInterface, Sequelize) => {
return queryInterface.removeColumn("Users", "firstname");
}
};
Running migrations again would only run the second one because it was saved in the database that the first one was already executed. But by default sequelize re-runs all seeders. So should I adjust the seeders/01-demo-user.js or change the default behavior and also store the seeders in the DB and create a new one that just updates the firstname?
What if firstname couldn't be null, then running migrations first and then the old version of seeders/01-demo-user.js would throw an error because firstname can't be null.
Re-running seeders leads to another problem: there is already a user with the demo#demo.com email. Running it a second time would duplicate the user. Or do I have to check for things like this in the seeder?
Previously, I just added the user-account in the migration so I could be sure when it was added to the DB and when I had to update it. But someone told me I was doing it all wrong and that I have to use seeders for tasks like this.
Any help/insights much appreciated.

In my opinion, a seeder is something, that is intended to run only once, while the migration is something that you add layer by layer to your DB structure continuously.
I would use seeders for populating some lookups or other data that most likely is not going to change, or test data. In the sequelize docs it's said, that "Seed files are some change in data that can be used to populate database table with sample data or test data."
If you want to make some dynamic data updates when data structure has already changed, you can run raw queries directly in your migrations if needed. So, if you, for instance, added some column in the up method, you can update the rows in the DB according to your business logic, e.g.:
// Add column, allow null at first
await queryInterface.addColumn("users", "user_type", {
type: Sequelize.STRING,
allowNull: true
});
// Update data
await queryInterface.sequelize.query("UPDATE users SET user_type = 'simple_user' WHERE is_deleted = 0;");
// Change column, disallow null
await queryInterface.changeColumn("users", "user_type", {
type: Sequelize.STRING,
allowNull: false
});
There is also an interesting discussion on this topic in Google Groups. I hope, this helps you.

In my experience migrations change structure. Seeders... seed data. Recently I was on a project that didn't have seeders configured. https://sequelize.org/master/manual/migrations.html#seed-storage. This will allow you to setup a file so your data isn't seeded more than once. Migration configuration is right there as well.

Hi try with bulkUpdate...
'use strict';
module.exports = {
up: (queryInterface, Sequelize) => queryInterface.addColumn(
'Users',
'user_type',
{
type: Sequelize.STRING,
allowNull: false,
}
).then(()=>
queryInterface.bulkUpdate('Users', {
user_type: 'simple_user',
}, {
is_deleted : 0,
})
),
down: (queryInterface, Sequelize) =>
queryInterface.removeColumn('Users', 'user_type')
};

Related

Unable to execute the raw query in Sequelize migrations

I am trying to update my database using Sequelize migrations so I have tried to write a Sequelize migrations like this
'use strict';
module.exports = {
up: (queryInterface, Sequelize, migration) => {
queryInterface.addColumn('coaching_class_entries', 'bill_cycle', {
type: Sequelize.INTEGER(4),
allowNull: false,
defaultValue: '0',
field: 'bill_cycle',
after: 'fees'
})
.then(() => queryInterface.addColumn('coaching_classes', 'bill_plans', {
type: Sequelize.JSON,
allowNull: false,
defaultValue: 'NULL',
field: 'bill_plans',
after: 'bill_cycle'
}))
.then(() =>
migration.migrator.Sequelize.query('UPDATE coaching_classes SET bill_plans = JSON_ARRAY(JSON_OBJECT("cycle", bill_cycle, "fee", fees));'));
},
down: (queryInterface, Sequelize) => {
let migrations = [];
migrations.push(queryInterface.removeColumn('coaching_class_entries', 'bill_cycle'))
migrations.push(queryInterface.removeColumn('coaching_classes', 'bill_plans'))
return Promise.all(migrations);
}
};
But it is always giving me error in raw query line
Cannot read property 'Sequelize' of undefined
What is the correct syntax for this?
I found it myself only we have to use simple queryInterface.sequelize.query

Sequelize, Issue with filtering on the join model for belongsToMany associations

The Sequelize docs suggest that you can filter your query on join table attributes using the following params on the options object:
[options.include[].through.where]
I've tried to use this formulation in the code below and found that the filtering does not work.
Model User and model Network are associated through the join table network-affiliations, which has additional attribute (boolean) 'confirmed'. I can't seem to write a query that returns only confirmed networks associated with a user.
My code is excerpted below.
const network_affiliations = db.define('network_affiliations', {
networkEmail: { type: Sequelize.STRING },
confirmed: { type: Sequelize.BOOLEAN }
}, {
freezeTableName: true,
defaultScope: {
where: {
confirmed: true
}
},
});
// User belongs to many Networks and Networks belong to many Users (join table)
User.belongsToMany(Network, { through: network_affiliations });
Network.belongsToMany(User, { through: network_affiliations });
//querying to find one user, including their confirmed networks
User.findOne({
where: {
email: req.body.email
},
include: [{ model: Network, through: { network_affiliations: { where: { confirmed: true } } } }]
})
I expect this query to return a user instance with its associated networks -- but only networks where confirmed: true. Instead I'm getting all networks associated with the user (including confirmed: false and confirmed: undefined).
As you can also see in the above code, I tried setting a defaultScope for the join table ({confirmed: true}). This also appears not to do anything.
I've also tried a User.findAll query that is otherwise identical, and that also does not work.
What am I missing, or Sequelize just not working here?
Sequelize version: "^3.30.4"

Define partial index in Sequelize migration?

I'm using the following index in my model definition right now:
{
name: 'unique_partner_id',
unique: true,
fields: ['partnerId'],
where: {
email: {
$ne: null
}
}
}
However, I want to use migrations instead of sync, so I'm trying to move this definition from model to the initial migration file.
There is a queryInterface.addIndex() method, however, I can't find any documentation for it.
SO, how do I define a partial index using queryInterface?
I was looking for how to do this in a model and your question answered that for me. So now I'm going to answer this for you! Here's the up function that worked for me:
up: function(queryInterface, Sequelize) {
return queryInterface.addIndex(
'tablename',
['column1', 'column2'],
{
name: 'indexname',
where: {column3: {[Sequelize.Op.ne]: null}}
}
);
}
Here's an example using the where querying syntax with operators:
up: (queryInterface, Sequelize) => queryInterface.addIndex(
'column_name',
['new_id'],
{
name: 'index_name',
where: { new_id: { [Sequelize.Op.ne]: null } },
},
)

Create Association in Sequelize to do a Left Join

I want to do a left join using Sequelize that does the same as this SQL code.
SELECT * FROM events LEFT JOIN zones ON event_tz=zid WHERE event_creator_fk=116;
I have two tables: events and zones (with a list of time zones).
When querying for all the events that are created by a specific user, I also want to get the name of the time zone and other details about the TZ.
I have tried many combinations of solutions by reviewing sample code, other Stack Overflow questions and the documentation as best I can. The query always works, but doesn't do any joins. That is, it below code always returns the list of events, but no time zone data from the zones table. The generated SQL is correct, except it doesn't have the ...LEFT JOIN zones ON event_tz=zid... part.
The below code is wrong. See answers for details.
db.Event.findAll(
{ where: { event_creator_fk: someUserID } },
{ include: [{ model: db.Zone } ]}
);
If I understand correctly, adding associations between tables in sequelize results in an additional column from automatically being created. This is not what I want to do. I do not want Sequelize to modify the tables or database in any way. I want to setup my database and it's tables without Sequelize. Therefore, I am not calling sequelize.sync(). I don't know if there is away to setup associations the way I want.
Model Definitions
module.exports = function (sequelize, DataTypes) {
var Event = sequelize.define('Event', {
eid: {
type: DataTypes.INTEGER,
primaryKey: true
},
event_tz: {
type: DataTypes.INTEGER,
primaryKey: true,
references: "Zone",
referencesKey: "zid"
},
}, {
classMethods: {
associate: function (models) {
return models.Event.hasOne(models.Zone);
}
},
freezeTableName: true,
timestamps: false,
tableName: 'events'
}
);
return Event;
};
module.exports = function (sequelize, DataTypes) {
return sequelize.define('Zone', {
zid: {
type: DataTypes.INTEGER,
primaryKey: true
}
}, {
classMethods: {
associate: function (models) {
return models.Zone.belongsTo(models.Event);
}
},
freezeTableName: true,
timestamps: false,
tableName: 'zones'
});
};
Table Definitions
DROP TABLE IF EXISTS zones;
CREATE TABLE IF NOT EXISTS zones (
zid integer NOT NULL,
country_code character(2) NOT NULL,
zone_name text NOT NULL,
PRIMARY KEY (zid)
);
DROP TABLE IF EXISTS events;
CREATE TABLE IF NOT EXISTS events (
eid BIGSERIAL NOT NULL,
event_tz SERIAL NOT NULL,
PRIMARY KEY (eid),
FOREIGN KEY (event_tz)
REFERENCES zones(zid) MATCH FULL ON DELETE RESTRICT
);
You need to reverse the associations and tell sequelize about your foreign key. belongsTo means 'add the fk to this model':
models.Event.belongsTo(models.Zone, { foreignKey: 'event_tz');
models.Zone.hasOne(models.Event, { foreignKey: 'event_tz');
// or hasMany if this is 1:m
Part of the problem was that I was using the findAll method incorrectly. The query options where and include should have been included as part of the same object. The first parameter to findAll is an options parameter. See here for more details. The correct code should look like the following.
db.Event.findAll(
{
where: { event_creator_fk: someUserID },
include: [{ model: db.Zone } ]
},
);

Writing Migrations with Foreign Keys Using SequelizeJS

The Background
I'm building a project with SequelizeJS, a popular ORM for NodeJS. When designing a schema, there appears to be two tactics:
Create model code and use the .sync() function to automatically generate tables for your models.
Create model code and write manual migrations using QueryInterface and umzug.
My understanding is that #1 is better for rapid prototyping, but that #2 is a best practice for projects that are expected to evolve over time and where production data needs to be able to survive migrations.
This question pertains to tactic #2.
The Question(s)
My tables have relationships which must be reflected through foreign keys.
How do I create tables with foreign key relationships with one another through the Sequelize QueryInterface?
What columns and helper tables are required by Sequelize? For example, it appears that specific columns such as createdAt or updatedAt are expected.
How do I create tables with foreign key relationships with one another through the Sequelize QueryInterface?
The .createTable() method takes in a dictionary of columns. You can see the list of valid attributes in the documentation for .define(), specifically by looking at the [attributes.column.*] rows within the params table.
To create an attribute with a foreign key relationship, use the "references" and "referencesKey" fields:
For example, the following would create a users table, and a user_emails table which references the users table.
queryInterface.createTable('users', {
id: {
type: Sequelize.INTEGER,
primaryKey: true,
autoIncrement: true
}
}).then(function() {
queryInterface.createTable('user_emails', {
userId: {
type: Sequelize.INTEGER,
references: { model: 'users', key: 'id' }
}
})
});
What columns and helper tables are required by sequelize? For example, it appears that specific columns such as createdAt or updatedAt are expected.
It appears that a standard model will expect an id, updatedAt, and createdAt column for each table.
queryInterface.createTable('users', {
id: {
type: Sequelize.INTEGER,
primaryKey: true,
autoIncrement: true
},
createdAt: {
type: Sequelize.DATE
},
updatedAt: {
type: Sequelize.DATE
}
}
If you set paranoid to true on your model, you also need a deletedAt timestamp.
I want to offer another more manual alternative because when using manual migrations and queryInterface I ran across the following problem: I had 2 files in the migration folder like so
migrations/create-project.js
migrations/create-projectType.js
because project had column projectTypeId it referenced projectType, which wasnt created yet due to the order of the files and this was causing an error.
I solved it by adding a foreign key constraint after creating both tables. In my case I decided to write it inside create-projectType.js:
queryInterface.createTable('project_type', {
// table attributes ...
})
.then(() => queryInterface.addConstraint('project', ['projectTypeId'], {
type: 'FOREIGN KEY',
name: 'FK_projectType_project', // useful if using queryInterface.removeConstraint
references: {
table: 'project_type',
field: 'id',
},
onDelete: 'no action',
onUpdate: 'no action',
}))
This is to create migration file for adding a column.
Here I want to add a column area_id in users table. Run command:
sequelize migration:create --name add-area_id-in-users
Once it gets executed creates a migration file timestamp-add-region_id-in-users in the migrations folder.
In the created migration file paste the below code:
'use strict';
module.exports = {
up: (queryInterface, Sequelize) => {
return Promise.all([
queryInterface.addColumn('users', 'region_id',
{
type: Sequelize.UUID,
references: {
model: 'regions',
key: 'id',
},
onUpdate: 'CASCADE',
onDelete: 'SET NULL',
defaultValue: null, after: 'can_maintain_system'
}),
]);
},
down: (queryInterface, Sequelize) => {
return Promise.all([
queryInterface.removeColumn('users', 'region_id'),
]);
}
};
Here in the users table I am going to create a column named region_id along with type and relation/foreign key/references. That's it.
So first one goes as the below solution and the second question: you need not explicitly mention createdAt and updatedAt because they are generated by the Sequelize for you.
The solution is:
queryInterface.createTable('Images', {
//...
}).then(
return queryInterface.addConstraint('Images', ['postId'], {
type: 'foreign key',
name: 'custom_fkey_images',
references: { //Required field
table: 'Posts',
field: 'id'
},
onDelete: 'cascade',
onUpdate: 'cascade'
})
)

Resources