knex js query many to many - node.js

i'm having trouble with node & knex.js
I'm trying to build a mini blog, with posts & adding functionality to add multiple tags to post
I have a POST model with following properties:
id SERIAL PRIMARY KEY NOT NULL,
name TEXT,
Second I have Tags model that is used for storing tags:
id SERIAL PRIMARY KEY NOT NULL,
name TEXT
And I have many to many table: Post Tags that references post & tags:
id SERIAL PRIMARY KEY NOT NULL,
post_id INTEGER NOT NULL REFERENCES posts ON DELETE CASCADE,
tag_id INTEGER NOT NULL REFERENCES tags ON DELETE CASCADE
I have managed to insert tags, and create post with tags,
But when I want to fetch Post data with Tags attached to that post I'm having a trouble
Here is a problem:
const data = await knex.select('posts.name as postName', 'tags.name as tagName'
.from('posts')
.leftJoin('post_tags', 'posts.id', 'post_tags.post_id')
.leftJoin('tags', 'tags.id', 'post_tags.tag_id')
.where('posts.id', id)
Following query returns this result:
[
{
postName: 'Post 1',
tagName: 'Youtube',
},
{
postName: 'Post 1',
tagName: 'Funny',
}
]
But I want the result to be formated & returned like this:
{
postName: 'Post 1',
tagName: ['Youtube', 'Funny'],
}
Is that even possible with query or do I have to manually format data ?

One way of doing this is to use some kind of aggregate function. If you're using PostgreSQL:
const data = await knex.select('posts.name as postName', knex.raw('ARRAY_AGG (tags.name) tags'))
.from('posts')
.innerJoin('post_tags', 'posts.id', 'post_tags.post_id')
.innerJoin('tags', 'tags.id', 'post_tags.tag_id')
.where('posts.id', id)
.groupBy("postName")
.orderBy("postName")
.first();
->
{ postName: 'post1', tags: [ 'tag1', 'tag2', 'tag3' ] }
For MySQL:
const data = await knex.select('posts.name as postName', knex.raw('GROUP_CONCAT (tags.name) as tags'))
.from('posts')
.innerJoin('post_tags', 'posts.id', 'post_tags.post_id')
.innerJoin('tags', 'tags.id', 'post_tags.tag_id')
.where('posts.id', id)
.groupBy("postName")
.orderBy("postName")
.first()
.then(res => Object.assign(res, { tags: res.tags.split(',')}))
There are no arrays in MySQL, and GROUP_CONCAT will just concat all tags into a string, so we need to split them manually.
->
RowDataPacket { postName: 'post1', tags: [ 'tag1', 'tag2', 'tag3' ] }

The result is correct as that is how SQL works - it returns rows of data. SQL has no concept of returning anything other than a table (think CSV data or Excel spreadsheet).
There are some interesting things you can do with SQL that can convert the tags to strings that you concatenate together but that is not really what you want. Either way you will need to add a post-processing step.
With your current query you can simply do something like this:
function formatter (result) {
let set = {};
result.forEach(row => {
if (set[row.postName] === undefined) {
set[row.postName] = row;
set[row.postName].tagName = [set[row.postName].tagName];
}
else {
set[row.postName].tagName.push(row.tagName);
}
});
return Object.values(set);
}
// ...
query.then(formatter);
This shouldn't be slow as you're only looping through the results once.

Related

Storing and querying JSON arrays in Redisjson with nodejs

What I was hoping to do was store an array of objects using RedisJSON very simply and then query that array.
I have something similar to this:
const data = [
{
_id: '63e7d1d85ad7e2f69df8ed6e',
artist: {
genre: 'rock',
},
},
{
_id: '63e7d1d85ad7e2f69df8ed6f',
artist: {
genre: 'metal',
},
},
{
_id: '63e7d1d85ad7e2f69df8ed6g',
artist: {
genre: 'rock',
},
},
]
then I can easily store and retrieve this:
await redisClient.json.set(cacheKey, '$', data)
await redisClient.json.get(cacheKey)
works great. but now I want to also query this data, I've tried creating an index as below:
await redisClient.ft.create(
`idx:gigs`,
{
'$.[0].artist.genre': {
type: SchemaFieldTypes.TEXT,
AS: 'genre',
},
},
{
ON: 'JSON',
PREFIX: 'GIGS',
}
)
and when I try and search this index what I expect is it to return the 2 documents with the correct search filter, but instead it always returns the entire array:
const searchResult = await redisClient.ft.search(`idx:gigs`, '#genre:(rock)')
produces:
{
total: 1,
documents: [
{ id: 'cacheKey', value: [Array] }
]
}
I can't quite work out at which level I'm getting this wrong, but any help would be greatly appreciated.
Is it possible to store an array of objects and then search the nested objects for nested values with RedisJSON?
The Search capability in Redis stack treats each key containing a JSON document as a separate search index entry. I think what you are doing is perhaps storing your whole array of documents in a single Redis key, which means any matches will return the document at that key which contains all of your data.
I would suggest that you store each object in your data array as its own key in Redis. Make sure that these will be indexed by using the GIGS prefix in the key name, for example GIGS:63e7d1d85ad7e2f69df8ed6e and GIGS:63e7d1d85ad7e2f69df8ed6f.
You'd want to change your index definition to account for each document being an object too so it would look something like this:
await redisClient.ft.create(
`idx:gigs`,
{
'$.artist.genre': {
type: SchemaFieldTypes.TEXT,
AS: 'genre',
},
},
{
ON: 'JSON',
PREFIX: 'GIGS:',
}
)
Note I also updated your PREFIX to be GIGS: not GIGS - this isn't strictly necessary, but does stop your index from accidentally looking at other keys in Redis whose name begins GIGS<whatever other characters>.

Sequelize join when key is inside a JSONB field

Is there a way to use include (which is actually a join table) to another Model, where the key is INSIDE a JSONB field? for example:
Item { id: INTEGER, someJsonbField: JSONB }
(item example: { id: 1, someJsonbField: { storeId: 2 } })
Then, for getting all of the items of store with id 2, you write something like this:
Item.findAll({ include: { model: 'Store', key: 'someJsonbField.storeId', ... } })
OFCOURSE, in a real world scenario, storeId should be inside Item directly, but only for the purpose of this question - How could it be done?

Node - Knex to return array of objects as a result of joined tables

I am using knexjs and node and running postgres db with this tables: menu and menuItem having one-to-many relationship. I found a solution here knex: what is the appropriate way to create an array from results? however this returns an array of strings. What i need is to return an array of objects and an empty array if null that looks exactly as the example below:
[
{
id: 123,
name: 'Sunday Menu',
items: []
},
{
id: 456,
name: 'Monday Menu',
items: [
{
id: 987,
name: 'Fried Chicken',
pcs: 69
},
{
id: 876,
name: 'Egg Soup',
pcs: 50
},
]
}
]
My menu and menuItem table schema looks similar to this:
menu_table: {
id,
name,
timestamps
}
menuItem_table: {
id,
menu_id,
name,
pcs,
timestamps
}
Currently, my code is like this:
knex('menu').leftJoin('menuitem', 'menu.id', 'menuitem.menu_id')
.select(['menu.id as menuID', knex.raw('ARRAY_AGG(menuitem.name) as items')])
.groupBy('menu.id')
And here's the result:
[
{
"menuID": "20091fff-ca8b-42d6-9a57-9f6e1922d0fa",
"items": [
null
]
},
{
"menuID": "2ddad4fa-7293-46c5-878f-cb2881be3107",
"items": [
"Fried Chicken",
"Egg Soup",
"Vegetable Dish"
]
}
]
UPDATE: I found out how to do it using raw query how ever i can't translate it using knex. Here's my code:
SELECT menu.*, COALESCE(menuitem.items, '[]') AS items FROM menu LEFT JOIN LATERAL (
SELECT json_agg(menuitem.*) AS items FROM menuitem WHERE menu.id = menuitem.menu_id
) menuitem ON true
I finally found a solution to my question. Since I was able to get my desired result thru raw query, i just translate it to knex. My final code is this:
const coalesce = knex.raw(`coalesce(menuitem.items, '[]') as items`)
const sub = knex('menuitem').select(knex.raw('json_agg(menuitem.*) as items')).whereRaw('menu.id = menuitem.menu_id')
return knex('menu').select(['menu.*', coalesce]).joinRaw('left join lateral ? menuitem on true', sub)
I'm gonna go with this code in the mean time until someone would give me the most accurate answer.
This work, im add attachments, ty DevWannabe :
const results = await knex.column({
'id': 'str.id',
'projectName': 'str.project_name',
})
.from('venturedoor.startups as str')
.leftJoin('venturedoor.attachments as att', 'str.id', 'att.startup_id')
// join array attachments
.select(['str.id', knex.raw('ARRAY_AGG(att.*) as attachments')])
.groupBy('str.id')
knex.raw('coalesce(usr.userName,usr.email ,seat.username) as username')
We can use like that measn. If userName is empty so will call email if email also then will call name from seats.

How to do a "keys-only-query" in Google Cloud Datastore (Node.js)

I am trying to do a "keys-only query" with Google Datastore API for Node.js, as exemplified in the documentation here.
I do that after having saved a number of records like this:
datastore.save(
records.map(
(record) => {
return {
key: datastore.key([kind, record.id]),
data: record,
};
}
)
The constant kind is a string. Each record has a valid unique id property (a number), which, as shown here, should also serve as the datastore key identifier.
The records are stored correctly. I can retrieve them all without problems through
datastore.runQuery(datastore.createQuery(kind))
.then( (results) => {
// whatever...
}
All the saved records are returned correctly.
But when I do a "keys-only query" like this (and as exemplified in the documentation):
const query = datastore.createQuery(kind)
.select('__key__');
datastore.runQuery(query)
.then( (results) => {
// whatever...
}
my results[0] return value is simply an array of empty objects like this:
results[0]: [ {}, {}, {}, {}, {}, ..., {}]
The number of empty objects returned here is the correct number of records of the given kind. But the problem is that they are empty objects. I expected to get the datastore key for each record here.
If, on the other hand, I do a "normal" projection query, on a "normal" property (like "id" - which should be identical with the datastore key, as far as I understand, after having defined the key through datastore.key[kind, record.id]), I retrieve the projected "id" properties correctly thus:
const query = datastore.createQuery(kind)
.select('id');
datastore.runQuery(query)
.then( (results) => {
// whatever...
}
Result:
results[0]: [
{ id: 5289385927 },
{ id: 5483575687 },
{ id: 5540575111 },
{ id: 5540622279 },
// ... and so on
]
So what is wrong with my "keys-only-query"? I have done it exactly in the way the documentation describes. But I get only empty results.
NOTE: I have tested this only in Datastore emulator. Same result in Datastore Emulator as in AppEngine.
The objects are not empty but contain only datastore keys, which are stored under a symbol property: datastore.KEY. In javascript, symbol properties might not output by default.
You can get entity key using symbol datastore.KEY
var keys = results.map(function(res) {
return res[datastore.KEY];
});

Using an arbitrary number of query params to filter results in mongoose

I'm building an API using node express and mongodb, with mongoose.
I have a post resource that handles user posts, and would like to be able to perform various queries on the post resource.
For instance I have a functions as that returns all posts as follows:
// Gets a list of Posts
exports.index = function(req, res) {
console.log(req.query);
Post.findAsync()
.then(mUtil.responseWithResult(res))
.catch(mUtil.handleError(res));
};
I looking for a good way of processing any additional query params that might come with the request.
/posts will return all posts, but /posts?user=12 will return posts by user with id 12 and /posts?likes=12 will return posts with 12 or more likes.
How can I check for and apply the these query params to filter and return the results since they may or may not be present.
Thanks ;)
If user=12 means "users with id 12", how does likes=12 mean "likes greater than 12"? You need to be more descriptive with your queries. You can do that by passing an array of objects. Send your query in a way that can be interpreted like this:
var filters = [
{
param: "likes",
type: "greater"
value: 12
},
{
param: "user",
type: "equal",
value: "12"
}]
var query = Post.find();
filters.forEach(function(filter) {
if (filter.type === "equal") {
query.where(filter.param).equals(filter.value);
}
else if (filter.type === "greater") {
query.where(filter.param).gt(filter.value);
}
// etc,,,
})
query.exec(callback);

Resources