this is is little bit weird. Sequelize query stopped working after... 1 year of functioning.
const where = { ... };
const attributes = ['id', 'code']; // This one making problems
const include = [...];
return SiteDao.findAll({ where, include, attributes });
I really didn't understand this because everything worked well, even try to logg queries and try them directly on DB, and they worked. Then I found that attributes broke it... But why? This is working for me
const where = { ... };
const attributes = { include: ['id', 'code'] }; // This one works for me
const include = [...];
return SiteDao.findAll({ where, include, attributes });
I don't really get it, because official documentation using the first version of attributes
My question is, why this happened and is it possible to somehow fix it, it's like every other api is using projection on attributes....
I think this documentation is a bit out of date or i suspect this works for an earlier version of sequelize than the one you are using. The official documentation explains how both options can be used. https://sequelize.org/master/class/lib/model.js~Model.html#static-method-findAll
Both
const attributes = ['id', 'code'];
const attributes = { include: ['id', 'code'] };
Should have been valid options. I suspect the ['id', 'code'] is translated in sql like column 'id' as 'code' cause there is an option to change the name of the attributes. You should try
const attributes = [['id','id'], ['code','code']];
If you want to keep your code consistent for the clarity.
Related
I'm working with typeorm + mongodb in one of my projects, and was reading the docs, in which I needed to do a find checking against a string array.
This should be simply achievable with a query {where: { property: {$in: myStringArray}}}
As documentation explains:
But for some reason, whenever I try to use this on code:
as I went through the types inside my code editor, it says that were expect strings or typeorm find methods(such as In, LessThan, ..., but those don't work with mongodb), but not objects.
Why the heck there's these suggestions on typeorm documentation, but inside code it looks different?
Anyone knows how to overcome this?
Update: This looks like an error that is still open in Typeorm project.
I found two ways of overcoming it. You can either:
Solution 1, add #ts-ignore:
// eslint-disable-next-line #typescript-eslint/ban-ts-comment
// #ts-ignore
With this, typescript won't complain about the typing, and will work as expected. You must use right above your where clause, such as:
const assetsExists = await assetRepository.find({
// eslint-disable-next-line #typescript-eslint/ban-ts-comment
// #ts-ignore
where: {
sourceId: { $in: assetIds },
},
})
Solution 2, you can cast it to unknown and then to string(ugly af, but it works)
const assetsExists = await assetRepository.find({
where: {
sourceId: { $in: assetIds } as unknown as string,
},
})
Personally I'm moving with the solution 1, and from what I check from open issues, this been appearing quite a lot, so if you have this problem, maybe consider one of the two approaches. :)
Background
Error: Order must be type of array or instance of a valid sequelize method
I thought I've solved this issue but turns out that I mess up with this error again.
I'm trying to make class method to calculate some properties for a model.
Let this be Model A.
Model A is associated with Model B. Because I have issue for generating proper column name for Model B when using array parameter.
I'm planning to bypass this issue by using sequelize.literal()
Pattern
Make a class method for Model A (a lot of business layer is used this function. So I can't take this away)
Prototype of this method is Model.function(options). This options object is validated inside of method function and if it needed mutated somehow.
Validated option object is passed to Model.findAll(options)
I'm impletmenting this solution as like code below
Router
const sequelize = require('sequelize')
const { ModelA } = require('../models')
...
router.get('/', ..., async (req, res, next) => {
try {
...
if(page < lastpage + 1){
const products = await ModelA.classMethod({
subquery : false,
include: [
...
],
...
order : sequelize.literal(`"Product.rating" DESC`)
})
...
}
...
} catch (e) {
...
}
})
Class method
ModelA.classMethod = async function(options){
const sequelize = require('sequelize')
const { ModelB } = require('.')
let { include, where, order, limit, offset, product } = options
...
const items = await ModelA.findAll({
subquery: false,
include: [
{
model: ModelB,
as: 'ModelB',
required: true,
where: product.where,
include: include
}
],
where: where,
limit: limit,
order: order
})
...
}
Weird thing is happening here. While passing parameter (Pattern 3), I got an error Error: Order must be type of array or instance of a valid sequelize method and this error seems that because the option passed is invalid sequelize.literal()
But actually what I passed is just sequelize.literal(`'Product.name' DESC`), no mutation in here.
So I tried to figure out what's wrong with my literal.
let { order } = option
console.log(order)//Literal { val: "'Product.rating' DESC" }
console.log(order instanceof sequelize.Utils.SequelizeMethod)//false
if(!order) order = null
console.log(order)//Literal { val: "'Product.rating' DESC" }
ModelA.findAll({ ..., order : order })
console.log('good!!!')//I want to see this log
Order itself looks fine but I think somewhere of prototype is broken.
The most weired part is if I replace the order with sequelize.literal(`'Product.name' DESC`)
,which is the same as what I passed into classMethod parameter, some kind of magic happens and error is gone.
const sequelize = require('sequelize')
let order = sequelize.literal("'Product.rating'")
console.log(order)//Literal { val: "'Product.rating' DESC" }
console.log(order instanceof sequelize.Utils.SequelizeMethod)//true!!
if(!order) order = null
ModelA.findAll({ ..., order : order })
console.log('good!!!')//I see this log and I can finally rest in peace.
If anyone has similar problem like me, would you please share some insight to solve this problem? So far I tried like below.
Passing router sequelize instance to class method. console.log(order instanceof sequelize.Utils.SequelizeMethod)//true so seems not broken actual query is not executed somehow.
Statically add order : sequelize.literal("'Product.rating' DESC") : work perfect but useless in production. This option should be dynanic so that user can control it.
My bad What were weird was not the error but me using different version between applications and project root. This problem is caused by version issue with nested directory.
/* MY PROJECT STRUCTURE */
project
|
|--api (sequelize 5.22)
|
|--auth
|
|--batch
|
|-- models (sequelize 6.3)
Recently, I upgraded sequelize from 5.22 to 6.3 while doing that, I missed upgrading sequelize of the applications. So the version between root and applications were being different. Since migration to typescript is on going in sequelize, sequelize inner types of 6.0 and 5.22 must be different.
Error: Order must be type of array or instance of a valid sequelize method
So that, this error was reasonable enough but I couldn't get that at the moment. There are many advices not to install same module in nested directory. I should've listened to them carefully.
But still, ordering with array (ex > [model, 'column', 'type']) gives me wrong query and ended up with unknown column error.
I am struggling to find the solution for that.
I want to have users which can belong to many organizations.
Each user can have a different role (I would prefer even roles but it sounds even more complicated...) at a specific organization.
In the table like User_Organization_Role I need to have fields like role (roleId?), isActive. Maybe some more.
I am using Feathers Plus generator but I do not think it matters in this case, however it may be beneficial to add something to the schema file?
I thought having belongsTo with simple organizationId field will be sufficient but I've realized that changing that to manyToMany, later on, would be painful so I think it is much better to implement that now.
I will appreciate any solutions / suggestions / best practices etc.
n:m relations are by far the most difficult to handle, and there's really no one-size-fits-all solution. The biggest thing is to read and understand this page and its sub-pages, and then read them 2 more times for good measure. Try to focus on doing one thing at a time. I outline how I would approach this with feathersjs in this issue:
https://github.com/feathersjs/feathers/issues/852#issuecomment-406413342
The same technique could be applied in any application... the basic flow goes like this:
Create or update your primary objects first (users, organizations, roles, etc.). There are no relations made at this point. You need to have your objects created before you can make any relations.
Create or update the relations. This involves updating a "join" table (aka: "mapping" or "through" table) with data from step #1. The join table can (and should) have its own model. It should contain a foreign key for each of the objects you are associating (userId, organizationId, roleId etc.). You can put other fields in this table too.
Here is some pseudo code for how I would define my models (only showing relevant code for brevity). There is a little more to it than what I describe below, but this should get you started.
const UserOrganizationRole = sequelize.define('User_Organization_Role', {
// Define any custom fields you want
foo: DataTypes.STRING
})
// Let sequelize add the foreign key fields for you.
// Also, save a reference to the relationship - we will use it later
User.Organization = User.belongsToMany(Organization, { through: UserOrganizationRole });
User.Role = User.belongsToMany(Role, { through: UserOrganizationRole });
Organization.User = Organization.belongsToMany(User, { through: UserOrganizationRole });
Role.User = Role.belongsToMany(User, { through: UserOrganizationRole });
... and here is how I would go about handling inserts
const user = await User.create({ ... });
const org = await Organization.create({ ... });
const role = await Role.create({ ... });
await UserOrganizationRole.create({
userId: user.id,
organizationId: org.id,
roleId: role.id,
foo: 'bar'
});
... and finally, load the data like so:
// Now we can reference those relationships we created earlier:
const user = await User.findById(123, {
include: [User.Organization, User.Role]
});
const org = await Organization.findById(456, {
include: [Organization.User]
});
I have two models that are related, Customers and Addresses. I first discovered this issue when I was trying to create a customer with a related address. For our purposes, a single customer can have multiple addresses, and when creating a new customer, we want to create an address at the same time as we create the customer.
I did some digging through the documentation and set up the relationship as best as I could, and this seemed to work well enough, but then I noticed that when I included both the models in modules together, (i.e. my routes/controllers), I was getting circular references.
Long story short, my research lead me to add the registry plugin to my bookshelf.js file. This worked at the time, but now it looks like my Address model isn't properly exported when being referenced in Customers.
Here's a snippet of my current configuration
// bookshelf.js
const bookshelf = require('bookshelf')(knex);
bookshelf.plugin([
'registry',
]);
module.exports = bookshelf;
// customers.js
const bookshelf = require('../bookshelf');
const Address = require('./address');
const Customer = bookshelf.Model.extend({
tableName: 'customers',
addresses: function () {
return this.hasMany('Address');
},
}, {
customCreate: function (attributes) {
return this.forge(attributes)
.save()
.tap(c => {
return Address.forge(attributes)
.save({
customer_id: c.get('id'),
});
})
}
});
module.exports = bookshelf.model('Customer', Customer);
// address.js
const bookshelf = require('../bookshelf');
const Customer = require('./customer');
const Address = bookshelf.Model.extend({
tableName: 'addresses',
customer: function () {
return this.belongsTo('Customer');
}
});
module.exports = bookshelf.model('Address', Address);
I started to notice that when I would run Customer.customCreate(), I got an error saying Address.forge is not a function. I threw some console logs into my customer.js file and saw that Address is an empty object ({}) when being referenced within customer.js. However, in other places, it's returning the proper Bookshelf model.
Looks to me like I'm trying to use my Address model in customers before it's properly required, which made me wonder if I'm structuring my project and models properly, or if there's any changes I need to make.
There's a circular reference problem alright. The best way to structure your models so that there are no such problems is to load them all during your app's initialization in a single file, e.g. index.js on your models' directory, attach each one to an object and export that object. Then you just require() that file and get access to all the models in a single place.
However, to solve your problem in a much easier way you just need to make a single change to your customCreate() method:
customCreate: function (attributes) {
return this.forge(attributes)
.save()
.tap(c => this.related('addresses').create(attributes))
}
}
This makes use of the Collection.create method to easily create a new model inside a collection, and since it's used on a relation it will also set the correct foreign key.
Note that the Registry plugin will not save you from circular dependency problems, but it will allow you to write your models in a way that avoids them.
It seems Mongoose is doing something really funky internally.
var Foo = new mongoose.model('Foo', new mongoose.Schema({a: String, b: Number}));
var foo = new Foo({a: 'test'; b: 42});
var obj = {c: 1};
foo.goo = obj; // simple object assignment. obj should be
// passed by reference to foo.goo. recall goo
// is not defined in the Foo model schema
console.log(foo.goo === obj); // comparison directly after the assignment
// => false, doesn't behave like normal JS object
Essentially, any time you try to deal with properties of a Mongoose model that aren't
a) defined in the model's schema or
b) defined as the same type (array, obj, ..) ... the model doesn't even behave like a normal Javascript object.
Switching line 4 to foo._doc.goo = obj makes the console output true.
edit: trying to reproduce weirdness
example 1:
// Customer has a property 'name', but no property 'text'
// I do this because I need to transform my data slightly before sending it
// to client.
models.Customer.find({}, function(err, data) {
for (var i=0, len=data.length; i<len; ++i) {
data[i] = data[i]._doc; // if I don't do this, returned data
// has no 'text' property
data[i].text = data[i].name;
}
res.json({success: err, response:data});
});
_doc exist on the mongoose object.
Because mongooseModel.findOne returns the model itself, the model has structure (protected fields).
When you try to print the object with console.log it gives you only the data from the database, because console.log will print the object public fields.
If you try something like JSON.stringify then you get to see inside the mongoose model object. (_doc, state ...)
In the case where you want to add more fields in the object and it's not working
const car = model.findOne({_id:'1'})
car.someNewProp = true // this will not work
If later you set the property to the object car and you didn't specify in the Model Schema before then Mongoose model is validating if this field exists and if it's the valid type.
If the validation fails then the property will not be set.
Update
Maybe I misunderstood your original question, but now it looks like the nature of your question changed, so the below information isn't relevant, but I'm leaving it. :)
I tested your code and it works fine for me. Mongoose doesn't execute any special code when you set properties that aren't part of the schema (or a few other special properties). JavaScript currently doesn't support calling code for properties that don't yet exist (so Mongoose can't get in the way of the set of the goo property for example).
So, when you set the property:
foo.goo = { c: 1 };
Mongoose isn't involved. If your console.log was something other than the code you displayed, I could see that it might report incorrectly.
Additionally, when you send the results back as JSON, JSON.stringify is being called, which calls toString on your Mongoose Model. When that happens, Mongoose only uses the properties defined on the schema. So, no additional properties are being sent back by default. You've changed the nature of the data array though to directly point at the Mongoose data, so it avoids that problem.
Details about normal behavior
When you set the property goo using Mongoose, quite a few things happen. Mongoose creates property getters/setters via the Object.defineProperty (some docs). So, when you set the goo property, which you've defined as a [String], a few things happen:
Mongoose code is called prior to the value being set onto the object instance (unlike a simple JavaScript object)
Mongoose creates an array (optionally) to store the data (a MongooseArray) which will contain the array data. In the example you provided, since you didn't pass an array, it will be created.
Mongoose will attempt to cast your data to the right type
It will call toString on the data passed as part of the cast.
So, the results are that the document now contains an array with a toString version of the object you passed.
If you checked the contents of the goo property, you'd see that it's now an array with a single element, which is a string that contains [object Object]. If you'd picked a more basic type or matched the destination property storage type, you would see that a basic equality check would have worked.
you can use toJSON() instead of _doc
Try using lean
By default, Mongoose queries return an instance of the Mongoose Document class. Documents are much heavier than vanilla JavaScript objects, because they have a lot of internal state for change tracking. Enabling the lean option tells Mongoose to skip instantiating a full Mongoose document and just give you the POJO.
https://mongoosejs.com/docs/tutorials/lean.html
Had same problem. Instead of updating my model.
const car = model.findOne({_id:'1'})
let temp = JSON.stringify(car);
let objCar = JSON.parse(temp);
objCar.color = 'Red'; //now add any property you want
this solves my problem
I was stuck on this today... Drove me nuts. Not sure if the below is a good solution (and OP has mentioned it too), but this is how I overcame this issue.
My car object:
cars = [{"make" : "Toyota"}, {"make" : "Kia"}];
Action:
console.log("1. Cars before the color: " + car);
cars.forEach(function(car){
car.colour = "Black"; //color is NOT defined in the model.
});
console.log("2. Cars after the color: " + car);
Problematic console output:
1. Cars before the color: [{"make" : "Toyota"}, {"make" : "Kia"}];
2. Cars after the color: [{"make" : "Toyota"}, {"make" : "Kia"}]; //No change! No new colour properties :(
If you try to pass in this property that was undefined in the model, via doc (e.g. car._doc.color = "black"), it will work (this colour property will be assigned to each car), but you can't seem to access it via EJS (frontend) for some reason.
Solution:
(Again, not sure if this is the best way... but it worked for me): Add in this new property (colour) in the car model.
var carSchema = mongoose.Schema({
make: String,
color: String //New property.
})
With the model redefined, everything worked as normal / expected (no _doc 'hacks' needed etc.) and I lived another day; hope it helps someone else.
There is some weirdness with Mongoose models and you have to check that Mongoose doesn't already have a model created in it's models array.
Here is my solution:
import mongoose from 'mongoose';
createModel = (modelName="foo", schemaDef, schemaOptions = {})=> {
const { Schema } = mongoose;
const schema = Schema(schemaDef, schemaOptions);
const Model = mongoose.models[modelName] || mongoose.model(modelName, schema);
return Model;
}
I use my own mongoose model class and base class for my models. I made this and it should work for you.
For those using spread(...) and/ can't see a solution, here's an example of #entesar's answer
Instead of spread or ._doc in:
import User from "./models/user";
...
async function createUser(req, res) {
const user = await User.create(req.body);
res.status(201).json({
message: "user created",
data: {
...user // OR user._doc,
token: "xxxxxxxx",
},
});
}
...
Use this
import User from "./models/user";
...
async function createUser(req, res) {
const user = await User.create(req.body);
res.status(201).json({
message: "user created",
data: {
...user.toJSON(),
token: "xxxxxxxx",
},
});
}
...
Ps: took me a while to understand the answer.
You should add .lean() on the find to have it skip all the Model "magic".