Why does Sequelize add extra columns to SELECT query? - node.js

When i want to get some records with joined data from the referenced tables, Sequelize adds the reference columns twice: the normal one and a copy of them, written just a little bit different.
This is my model:
module.exports = function(sequelize, DataTypes) {
return sequelize.define('result', {
id: {
type: DataTypes.INTEGER(10),
allowNull: false,
primaryKey: true,
autoIncrement: true
},
test_id: {
type: DataTypes.INTEGER(10),
allowNull: false,
references: {
model: 'test',
key: 'id'
}
},
item_id: {
type: DataTypes.INTEGER(10),
allowNull: false,
references: {
model: 'item',
key: 'id'
}
},
}, // and many other fields
{
tableName: 'result',
timestamps: false, // disable the automatic adding of createdAt and updatedAt columns
underscored:true
});
}
In my repository I have a method, which gets the result with joined data. And I defined the following associations:
const Result = connection.import('../../models/storage/result');
const Item = connection.import('../../models/storage/item');
const Test = connection.import('../../models/storage/test');
Result.belongsTo(Test, {foreignKey: 'test_id'});
Test.hasOne(Result);
Result.belongsTo(Item, {foreignKey: 'item_id'});
Item.hasOne(Result);
// Defining includes for JOIN querys
var include = [{
model: Item,
attributes: ['id', 'header_en']
}, {
model: Test,
attributes: ['label']
}];
var getResult = function(id) {
return new Promise((resolve, reject) => { // pass result
Result.findOne({
where: { id : id },
include: include,
// attributes: ['id',
// 'test_id',
// 'item_id',
// 'result',
// 'validation'
// ]
}).then(result => {
resolve(result);
});
});
}
The function produces the following query:
SELECT `result`.`id`, `result`.`test_id`, `result`.`item_id`, `result`.`result`, `result`.`validation`, `result`.`testId`, `result`.`itemId`, `item`.`id` AS `item.id`, `item`.`title` AS `item.title`, `test`.`id` AS `test.id`, `test`.`label` AS `test.label` FROM `result` AS `result` LEFT OUTER JOIN `item` AS `item` ON `result`.`item_id` = `item`.`id` LEFT OUTER JOIN `test` AS `test` ON `result`.`test_id` = `test`.`id` WHERE `result`.`id` = '1';
Notice the extra itemId, testId it wants to select from the result table. I don't know where this happens. This produces:
Unhandled rejection SequelizeDatabaseError: Unknown column 'result.testId' in 'field list'
It only works when i specify which attributes to select.
EDIT: my tables in the database already have references to other tables with item_id and test_id. Is it then unnecessary to add the associations again in the application code like I do?
A result always has one item and test it belongs to.
How can i solve this?
Thanks in advance,
Mike

SOLUTION:
Result.belongsTo(Test, {foreignKey: 'test_id'});
// Test.hasMany(Result);
Result.belongsTo(Item, {foreignKey: 'item_id'});
// Item.hasOne(Result);
Commenting out the hasOne, hasMany lines did solve the problem. I think I messed it up by defining the association twice. :|

Sequelize uses these column name by adding an id to the model name by default. If you want to stop it, there is an option that you need to specify.
underscored: true
You can specify this property on application level and on model level.
Also, you can turn off the timestamps as well. You need to use the timestamp option.
timestamps: false

Although your solution fixes your immediate problem, it is ultimately not what you should be doing, as the cause of your problem is misunderstood there. For example, you MUST make that sort of association if making a Super Many-to-Many relationship (which was my problem that I was trying to solve when I found this thread). Fortunately, the Sequelize documentation addresses this under Aliases and custom key names.
Sequelize automatically aliases the foreign key unless you tell it specifically what to use, so test_id becomes testId, and item_id becomes itemId by default. Since those fields are not defined in your Result table, Sequelize assumes they exist when generating the insert set, and fails when the receiving table turns out not to have them! So your issue is less associating tables twice than it is that one association is assuming extra, non-existing fields.
I suspect a more complete solution for your issue would be the following:
Solution
Result.belongsTo(Test, {foreignKey: 'test_id'});
Test.hasMany(Result, {foreignKey: 'test_id'});
Result.belongsTo(Item, {foreignKey: 'item_id'});
Item.hasOne(Result, {foreignKey: 'item_id'});
A similar solution fixed my nearly identical problem with some M:N tables.

Related

Sequlize generate a table with a foreign key while explicitly requested not to

I have the following association in PersonEntity:
#HasMany(() => SocialPostEntity, {
foreignKey: 'parentId',
foreignKeyConstraint: false,
})
posts?: SocialPostEntity[];
On the other hand the related column on SocialPostEntity:
#Column(DataType.INTEGER)
parentId: number;
The generated create table script has created the parentId column with a foreign key to PersonEntity. event though I have explicitly mentioned foreignKeyConstraint: false
Im not intended to use synchronize: true in production but I'm afraid that it means that my mapping is not accurate.
Bottom line I want parentId to be a simple integer, that's it.
Thanks

Sequelize: BelongsToMany with Polymorphic Relationships fails on scope after first (successful) try

We are running:
NodeJS v13.11
Sequelize v5.22.3
Postgres v11.7-2
I have 3 models, GameVersion, Tag, and TagTaggable (representing the associative entity). A GV can have many Tags, and a Tag can be associated with many GVs (or any other model via TagTaggable).
Models are built with this (not showing the hooks to handle and clean-up after the polymorphism):
GameVersion:
this.belongsToMany(models.Tag, {
through: {
model: models.TagTaggable,
unique: false,
scope: {
taggableType: 'game_version',
},
},
as: 'tags',
foreignKey: 'taggable_id',
constraints: false,
});
this.hasMany(models.TagTaggable, {
foreignKey: 'taggable_id',
scope: {
taggableType: 'game_version',
},
});
Tag:
this.belongsToMany(models.GameVersion, {
through: {
model: models.TagTaggable,
unique: false,
scope: {
taggableType: 'game_version',
},
},
as: 'gameVersions',
foreignKey: 'tag_id',
constraints: false,
});
this.hasMany(models.TagTaggable, {
scope: {
taggableType: 'game_version',
},
});
TagTaggable:
this.belongsTo(models.Tag, {
as: 'tag',
foreignKey: 'tag_id',
});
// Polymorphic relationships
this.belongsTo(models.GameVersion, {
foreignKey: 'taggable_id',
constraints: false,
as: 'gameVersion',
});
The Tags are applied to a GV by:
await gv.setTags(metadata.tags);
where metadata.tags is a collection of Tag models.
This works perfectly, the first time I run it, and I can see that, debugging into the bowels of the Sequelize BelongsToMany.updataAssociations method, the scope is correct on that first time, by looking at this.through.scope on the BelongsToMany object.
I get:
{taggableType: 'game_version'}
Notice the camel-cased key, which is what it should be.
This results in the following query:
INSERT INTO "tag_taggable" ("taggable_type","created_at","updated_at","taggable_id","tag_id") VALUES ('game_version','2020-11-27 18:49:34.767 +00:00','2020-11-27 18:49:34.767 +00:00',82,29)
The problem arises on any subsequent attempt (meaning it works the first time after starting the server, but any attempts prior to re-starting the server fail), and I can see that this.through.scope now results in:
{taggable_type: 'game_version'}
Notice the snake-cased key.
This results in the following query (notice the lack of the "taggable_type" column):
INSERT INTO "tag_taggable" ("created_at","updated_at","taggable_id","tag_id") VALUES ('2020-11-27 18:51:16.423 +00:00','2020-11-27 18:51:16.423 +00:00',83,29)
and throws a "not-null constraint violation."
This SEEMS to be down in the guts of Sequelize, but I cannot imagine it not being surfaced already if it were (unless polymorphic m:n is really uncommon).
Has anyone had this experience, and/or can anyone shed any light on what is going on here?
I would really like to use the setTags magic method, but I am at the point of just hand-building the TagTaggable objects and stuffing them in the DB myself.
TIA for any insights/assistance,
Chris

How to get sequelize getter to work with findAll

For sake of example, I have this very simple model:
const Blog = sequelize.define(
'Blog',
{
id: {
type: DataTypes.STRING,
primaryKey: true,
allowNull: false,
get() {
return this.getDataValue('id').toString();
}
},
},
)
I would like to return the id my records as strings, but it does not work:
const blogs = Blog.findAll({ raw: true })
console.log(blogs)
Returns a list of blogs with the id property as a number.
Any ideas why and if that's not the correct way to do this, what would it be?
If you want to use a different type for the primary key make sure to set autoIncrement: false. Typically you would use either a DataTypes.INTEGER or DataTypes.UUID as a primary key type but should work with a DataTypes.String as well. You will need to insert a unique value for each row.
If you want to run the getter you need to remove raw: true. When you specify raw results the columns that are returned from the database are mapped directly to a JSON object, in this case they are numeric. If you want to map the results back to a plain JSON object instead of Model Instances you can use Array.map() and Instance.toJSON().
Note that in your example you omitted await or thenable syntax as well which is necessary to get the results asynchronously.
const blogs = await Blog.findAll();
const plainObjs = blogs.map((blog) => blog.toJSON());

Am i supposed to keep Sequelize models and migrations in sync?

I'm new to Sequelize.js and Databases in general, i haven't used migrations before, but i know that they can be used to make changes to the tables structure in a non-destructive way.
However i'm not sure where to declare column options (notNull, references, validate, ENUM values, etc...)
Should i declare such options in the the model file, or migration file? or both?
Wouldn't adding the options to both model and migration cause duplicate code?
(keep in mind that i'm talking about the initial migrations that create tables to the database, not the migrations that add columns and stuff...)
Any help would be appreciated!
I see three options you can take. The first two options might be edge cases but it helps to understand.
Destructive option
You want to prototype a project and you don't mind losing your data, then you could potentially do not care about migration files and synchronize your database according to your model with:
await sequelize.sync({ force: true });
It will execute on all your models:
DROP TABLE IF EXISTS "your_model" CASCADE;
CREATE TABLE IF NOT EXISTS "your_model" (...)
This command can be executed at the start of your application for example.
 Static option
As you mentioned you don't want to add columns and stuff, it's probably a good option.
Now if you don't want to lose data, you could simply use the sync method without the force option:
await sequelize.sync({ });
It will only generate:
CREATE TABLE IF NOT EXISTS "your_model" (...)
Hence, your tables are created according to your models and you don't have to create migration files.
However, if you want to modify your model and it's the most frequent usecase, the new column will not be generated in the table dynamically that's why you need migration scripts.
Flexible option
You will have to define both migration file and your model. That's what the cli does. Here is an example:
# npx sequelize-cli init or create migrations and models folders
npx sequelize-cli model:generate --name User --attributes firstName:string,email:string
Now you will have two more files:
// migrations/<date>-create-user.js
module.exports = {
up: (queryInterface, Sequelize) => {
return queryInterface.createTable('Users', {
id: {
allowNull: false,
autoIncrement: true,
primaryKey: true,
type: Sequelize.INTEGER
},
firstName: {
type: Sequelize.STRING
},
email: {
type: Sequelize.STRING
},
createdAt: {
allowNull: false,
type: Sequelize.DATE
},
updatedAt: {
allowNull: false,
type: Sequelize.DATE
}
});
},
down: (queryInterface, Sequelize) => {
// I usually remove this and create the table only if not exists
return queryInterface.dropTable('Users');
}
};
// models/users.js
module.exports = (sequelize, DataTypes) => {
const User = sequelize.define('User', {
firstName: DataTypes.STRING,
email: DataTypes.STRING
}, {});
User.associate = function(models) {
// associations can be defined here
};
return User;
};
You could refactor the code from the migration and the model, however it will be rather cumbersome because some migration files will only add one column, so merging all of them into the model might probably be less clear.
You should do it in both because as time goes by your models and inital migration will differ from each other. So I suppose you should determine a final structure in models and after that create an initial migration.
Constraints are defined and run on SQL level while validations are run on application level. Sequelize supports having validations and constraints on Models, only constraints can be defined in migrations.
My opinion is to put all constraints in migrations, and validations in Models. That way you have some kind of separation of concern as validations are run before query is made to the database - where constraints are run. You can read more on Sequelize's Validations and Constraints Validations and Constraints

Node/Express Project: Create and Sync a "Join Class/Model" to a Postgres Database

I'm building a Node/Express/Postgres version of an app that I already built in Rails. I'm learning Node, so I figured I'd rebuild something that I know works.
For now, I'm dumping everything in one file (set up my database, defined my models, etc.), just to make sure I have everything set up correctly before I divvy them up into different files.
I set up my postgres database at the very top of the file, like so:
var Sequelize = require('sequelize');
var db = new Sequelize('my_database_name', 'my_username', null, {
host: 'localhost',
dialect: 'postgres',
});
With regard to my models, I have a Politician model:
var Politician = db.define("politician", {
name: {
type: Sequelize.STRING,
},
politicalParty: {
type: Sequelize.STRING
}
});
A Category model:
var Category = db.define("category", {
name: {
type: Sequelize.STRING
},
keywords: {
type: Sequelize.ARRAY(Sequelize.TEXT)
},
});
And a join model of Politician and Category, called "Interest". Because Interest is a join model, it will have a "politicianId" and "categoryId" properties....but will those properties automatically generate in the database? And so, is this how I would define the Interest model, with no properties?
Interest Model:
var Interest = db.define("interest")
Or, will I have to be specific, and create "politicianId" and "categoryId" properties? Like so:
Interest Model:
var Interest = db.define("interest", {
id: {
type: Sequelize.INTEGER,
primaryKey: true,
autoIncrement: true
},
categoryId: {
type: Sequelize.INTEGER,
foreignKey: true
},
politicianId: {
type: Sequelize.INTEGER,
foreignKey: true
}
});
Also, do I need the "foreignKey: true" bit? Or will it automatically know that those properties are foreign keys? Also, do I need the "id" property? I know models automatically create their own primary key "id"...but again, I've been at this for hours, looking at docs, and trying everything.
I then defined my associations (again, all of this is the same file):
Politician.belongsToMany(Category, {through: "Interest"});
Category.belongsToMany(Politician, {through: "Interest"});
The Node/Sequelize docs seems to suggest that defining those 2 associations above will automatically "create a new model called Interest with the equivalent foreign keys politicianId and categoryId." So, do I even need to define a "Interest" model? Also, do I need the follow associations to describe that Interest belongs to Politician and Category?
Interest.belongsTo(Politician);
Interest.belongsTo(Category);
If I don't write the associations saying that Interest belongs to Politican and Catetory, I don't get the "politicianId" and "categoryId" columns in the Interest table. Just the "id" and createdAt/updatedAt columns.
I then created an instance of Politician, Category, and Interest, to persist everything to the database, to see if everything is there and set up correctly:
Politician Object:
var politician1 = Politician.sync({force: true}).then(function(){
return Politician.create(aPoliticianObjectDefinedInthisFile);
});
This works perfectly. I see this object in the politician table in the database.
Category Object:
var category1 = Category.sync({force: true}).then(function(){
return Category.create(aCategoryObjectDefinedInThisFile);
});
This works perfectly. I see this object in the category table in the database.
Here is what doesn't work. Creating an instance/object of Interest and synching it to the database. My thinking is, if I put integers as values, it will know that "politicianId: 1" means point to the politician object with an id of 1, and the same for "categoryId: 1". But when I write it as I have it below, the Interest table doesn't even show up in the Postgres database at all.
Interest Object:
Interest.sync({force: true}).then(function(){
return Interest.create(
{
politicianId: 1,
categoryId: 1
}
);
});
However, when I create the object of Interest like this, with no properties defined, the Interest table appears in the database, along with the "politicianId" and "categoryId" columns, however, those columns are empty. The object's primary id is in there at 1, and the "createdAt" and "updatedAt" columns have data too. But the foreign key columns are blank.
Interest Object:
Interest.sync({force: true}).then(function()
{
return Interest.create(
{
// No properties defined.
}
);
}
);
Sorry for this long post, lol, but, in all:
Am I creating the "Interest" model correctly?
Am I writing the associations for "Interest" correctly?
Do I even need to write associations for Interest, if I already have associations for its parent classes, Politican and Category defined?
In my Rails app, my associations for Politican and Category are like so:
Politician has_many interests
Politican has_many categories through interests
Category has_many interests
Category has_many politicians through interests
Interest belongs_to politician
Interest belongs_to category
But I use the "belongsToMay" association in Node because I got an error telling me to do so.
Basically, I need to create an instance of Politician, an instance of Category, and an instance of Interest that has "politicianId" and "categoryId" columns that point to those aforementioned instances of those classes.
politicanABC -- id: 1
categoryABC -- id: 1
instanceABC -- id: 1; politicanId: 1 (referring to politicanABC); categoryid: 1 (referring to categoryABC).
My app is set up like that in Rails and works wonderfully.
Help and thank you in advance :-)
You don't have to define the Interest model if you are not going to add any additional fields. Sequelize will internally define the model and add all required fields once you do following:
Politician.belongsToMany(Category, {through: "Interest"});
Category.belongsToMany(Politician, {through: "Interest"});
Sync needs to run on database level and not on tables since Interest model is implicit at this point.
db.sync({force: true});
Sequelize will add relationship build methods on both Politician and Category instances. Category will have methods addPolitician(), addPoliticians([]), setPoliticians([]), getPliticians(). Politician instances will have similar functions to associate categories to them. You can connect these after create option is performed on both objects successfully.
Politician.create({name: 'John Doe', politicalParty: 'Nice Party'})
.then(function(politician) {
Category.create({name: 'Nicers'})
.then(function(category) {
politician.addCategory(category);
});
});
You can also search and associate existing items using helper methods. Alternatively you can associate objects manually by accessing db.models.Interest model and running creates on it.

Resources