We are using js-data-sql DSSqlAdapter in our backend nodejs service.
Our model definition has a hasOne Relationship defined as follows:
module.exports = {
name: 'covariance_predictions',
idAttribute: 'id',
relations: {
belongsTo: {
assets: {
localKey: 'asset_1_id',
localField: 'Covariance_Predictions'
}
},
hasOne: {
currencies: {
localField: 'currency',
localKey: 'currency_id'
}
}
}
};
We call the adapter using:
covariancePredictions.findAll(params, {
with: ['currencies']
})
Question
After enabling knex debugging, we figured out, that it does not use a left join statement, but a subsequent SQL query like:
sql: 'select "currencies".* from "currencies" where "id" in (?, ?, ?, ?, ?, ?, ?)' }
Does anyone has any idea how to make js-data-sql DSSqlAdapter build a left join instead? Like:
select "covariance_predictions".id, [...], "currencies".code from "covariance_predictions" left join "currencies" on "currencies".id = "covariance_predictions".currency_id;
I'm one of the maintainers of js-data-sql. This is currently not supported as all relations loaded via with are done so using loadingWithRelations which performs the subsequent select ... where "id" in (...) for each relation requested.
Loading hasOne and belongsTo as part of the original query should definitely be possible via a left outer join, but not an inner join as the loading of the original entity is not dependent on the existence of a relation(s).
I've created a github issue to track this change, although I'm not sure when I'll be able to make it as js-data-sql also needs to port over to extend js-data-adapter for js-data 3.0
Related
I have a sequelize query which renders the correct SQL statement with the exception that it will return the primary key of the joined table. As this query is using Group By the additional field is unacceptable but I am not certain how to remove it.
userDB.tblUsers.belongsTo(userDB.tblSMSSent, { foreignKey: 'collector', targetKey: 'user' });
userDB.tblSMSSent.hasMany(userDB.tblUsers);
let users = await userDB.tblUsers.findAll({
attributes: ['collector', 'tblUsers.id'],
include: {
model: userDB.tblSMSSent,
as: 'tblSMSSent',
attributes: [[sequelize.fn('COUNT', 'tblSMSSent.id'), 'numMessages']]
},
group: ['tblUsers.collector', 'tblUsers.id'],
logging: console.log
})
The SQL rendered is such:
SELECT
[tblUsers].[collector],
[tblUsers].[id],
[tblSMSSent].[id] AS [tblSMSSent.id],
COUNT(N'tblSMSSent.id') AS [tblSMSSent.numMessages]
FROM [dbo].[tblUsers] AS [tblUsers] LEFT OUTER JOIN [dbo].[tblSMSSent] AS [tblSMSSent] ON [tblUsers].[collector] = [tblSMSSent].[user]
GROUP BY [tblUsers].[collector], [tblUsers].[id];
I need this query without tblSMSSent.id included. I have tried using exclude: tblSMSSent.id as outlined in this article in the include attributes but that has not succeeded either. How am I able to correctly exclude this column from my query?
Added the raw:true tag in the JSON sent to sequelize. This forced only the inclusion of the required fields without primary keys.
Designing a PostgreSQL database that will be queried by a Node API using Sequelize. Currently, I have a table called recipes that has columns called ingredients and instructions. Those columns are stored for a given as an array of strings like {Tomatoes, Onions}.
That method of storage worked fine for simply fetching and rendering a recipe on the client side. But it wasn't working well for fuzzy search querying because, using Sequelize all I could do was ingredients: { [Op.contains] : [query] }. So if a user typed tomatoes there was no way to write a "fuzzy" search query that would return a recipe with an ingredient Tomatoes.
And then I read this in the PostgreSQL documentation:
Arrays are not sets; searching for specific array elements can be a sign of database misdesign. Consider using a separate table with a row for each item that would be an array element. This will be easier to search, and is likely to scale better for a large number of elements.
Now I'm considering storing ingredients and instructions as separate tables, but I have a couple of questions.
1) As a recipe can have multiple ingredients related to it, should I just use a foreign key for each ingredient and the Sequelize hasMany relationship? That seems correct to me, except that I'm now potentially duplicating common ingredients each time a new recipe is created that uses that ingredient.
2) What would be the best way to write a fuzzy search query so that a user could search the main columns of the recipes table (e.g. title, description) and additionally apply their query to the instructions and ingredients tables?
Essentially I'd like to end up with a fuzzy search query applied to the three tables that looks something like this...
const recipes = await req.context.models.Recipe.findAll({
where: {
[Op.or]: [
{ title: { [Op.iLike]: '%' + query + '%' } },
{ description: { [Op.iLike]: '%' + query + '%' } },
{ ingredients: { ingredient: { [Op.iLike]: '%' + query + '%' } } },
{ instructions: { instruction: { [Op.iLike]: '%' + query + '%' } } }
]
}
});
Thanks!
I have done this, i happen to use graphql in my node layer with sequelize, and i have filter objects that do this type of thing. You'll just need some include statements in your Recipie.findAll.. after your initial where clause where you evaluate whether you are searching title or description or both type thing. i sent my search params in with prefix's i could strip off that told me what sequelize op's i would want to use on them and just ran my args through a utility method to create my where clause, but i know there are many ways to skin that cat. i just did not want to clutter up my resolvers with tonnes of hardcoded ops and conditional clauses was all.... your include might look something like this
include: [{
model: models.Ingredient,
as: 'Ingredients',
through: { some join table specifying keys where necessary since this
is many to many }
where: {some conditional code around your search param},
}, {
model: models.Instruction,
as: 'Instructions',
where: {some conditional code around your search param},
}],
There is good documentation around multiple includes, or nested includes in the sequelize docs, but from what i see above you have a fairly good understanding of what you need to do. To uncomplicate things a bit, i'd start with just searching on your fields from recipie (title, description) before you add the includes and get that working, then it will be a little clearer how you want to form your where clauses.
alternativley.. you can skip the includes and write associations in your models and call them with getters and pass the where clauses to those... i do that as well and again well documented stuff now.. Sequelize has really upped their game
Recipie.associate = function (models) {
models.Recipie.hasMany(models.Ingredient, { as: 'Ingredients', through: "recipie_ingredient" foreignKey: 'recipie_id'});
};
now you have a getter for Ingredients, and if you declare belongsToMany targetting back at Recipie in the Ingredient model then you'll have a getter there as well, and you can pass your search string to that via where clause and get all recipies that have a given ingredient or ingredient list type thing.... Clear as mud?
It's possible that I'm not quite understanding how hash/primary keys work in DynamoDB, but I'm trying to create a model (using Serverless + Dynogels/NodeJS) for a messaging service.
The model looks like this:
const ConversationORM = dynogels.define('Conversation', {
hashKey: 'id',
timestamps: true,
tableName: config.CONVERSATION_TABLE,
schema: {
id: Joi.string(),
users: Joi.array(), // e.g. ['foo', 'bar', 'moo']
messages: Joi.array()
}
})
As you can see, users is an array, which lists the userIds of the conversation's participants.
I need to create a service which finds all conversations that a user is participating in. In MongoDB (which I'm far more familiar with), I'd do something like:
Conversation.find({users: {"$in": ['foo']} }).then(....
Is there something equivalent I can do in DynamoDB? This is an API call that will happen quite often so I'm hoping to make it as efficient as possible.
This answer takes into account a comment on Hunter Frazier's answer saying you don't want to use a Scan.
When using a Query you need specify a single partition key in the operation. In your schema this would mean partitioning on the userid attribute, which is a set. Partition keys in DynamoDB must be a top-level scalar attribute. As userid is not scalar (its a set), you cannot use this attribute as an index, and therefore you cannot do a Query for the conversations a user is part of.
If you need to do this Query, I would suggest revisiting your schema. Specifically I would suggest implementing the Adjacency list pattern which works well in databases containing many-to-many relationships.
You can see some additional notes on the article above I have written on this answer DynamoDB M-M Adjacency List Design Pattern
In your case you would have:
Primary Key: ConversationID
Sort Key: UserID
GSI Primary Key: UserID
You can then use the GSI Primary key in a Query to return all conversations the user is part of.
I'm not familiar with Dynogels or Serverless but if it uses the regular API this might work:
var params = {
ExpressionAttributeNames: {
"U": "users"
},
ExpressionAttributeValues: {
":a": {
S: "John Doe"
}
},
FilterExpression: "Author = :a",
ProjectionExpression: "#U",
TableName: "Conversations"
};
dynamodb.scan(params, function (err, data) {
if (err) console.log(err, err.stack);
else console.log(data);
});
I have gone through the Sequelize doc, but can't find anything helpful
What I want to do is to add raw query or custom model in include, is it possible ?
model.Post.findAll({
include: [{
model: model.User,
attributes: ['fullname', 'profilepic'],
},
{
model: model.PostComments,
},
{
model: "Raw Query"
}
]
}
What I want to achieve is like :
Select post_id, count(*) as total_likes from post_likes group by post_id
I can't achieve this by using simply include, so what I want to do is create a table/model from above query and then use it inside include.
If I use group by withing include it gives group by from top level, and I want to apply group by just for the post_like table.
Please let me know, if it found confusing or not clear.
I was looking to do the same, use a raw query dynamically formed inside the include but there's no possible way to use include without a model https://sequelize.org/master/class/lib/model.js~Model.html#static-method-findAll .
For the purposes of my MySQL I turned my inner join (include) into a where in. I was doing the inner join to avoid the exception This version of MySQL doesn't yet support 'LIMIT & IN/ALL/ANY/SOME subquery . I get this exception because I have a subquery with LIMIT applied.
If it helps anyone:
let rows = await j_model.findAll({
attributes: [...j_columns, `${association}.meta_key_id`],
include: [
{
model: um_model,
as: association,
attributes: ['um_id'],
on: {'j_id' : {$col: 'j.j_id'} }
],
where: {'j_id': {$in: sequelize.literal(`(SELECT * FROM (${massive_inner_raw_query}) as tmp)`)}},
logging: console.log
});
The actual magic is in this line:
where: {'j_id': {$in: sequelize.literal(`(SELECT * FROM (${massive_inner_raw_query}) as tmp)`)}}
The SELECT * FROM removes that exception and lets me do a where in instead of the wanted INNER JOIN. Maybe you can apply a similar deal to your problem.
You can use
Model.findAll({
attributes: [[models.sequelize.literal('CASE WHEN "field1" = true THEN 55
ELSE 23 END'), 'field3']]
}
OR
Model.findAll({
attributes: { include: [[models.sequelize.literal('CASE WHEN "field1" = true THEN 55 ELSE 23 END'), 'field3']]}
}
I am executing join query in node.js using sequelize . How to write join queries using sequelize ? thank you in advance. have a good day.
You first need to define associations between your models. If I have two models User and Customers, I can define and association like this
User.belongsTo(Customer);
Customer.hasMany(User);
Then when you query you can join by specifying include in the options
Customer.finOne({
where: {
id: 1,
},
include: {
model: User,
where: {
id: 4,
}
}
}