Recommended data-structure (for not querying nested data in node.js) - node.js

Mongo-DB doesn't support query with wildcards in a nested structure.
In a data-structure that looks like this:
Courses = [{
'name': 'Learning node.js in 1 day',
'roles': {
'team': { subscribed: [ 'User1' ] },
'participant': { subscribed: [ 'User1', 'User2' ] },
'host': { optional: true, subscribed: [] }
}
}]
We would need wildcard-lookup to find subscribers in different roles in order not to use a query like this:
{ $or : [
{"roles.team.subscribed": 'User1'},
{"roles.participant.subscribed": 'User1'}
{"roles.host.subscribed": 'User1'}
]}
Anyway this does not work if we have an open list of roles.
And if we change it to something like this:
'roles': ['team', 'participant', 'host'],
'subscribed': [
{'user':'User1', 'roles': ['team', 'participant']},
{'user':'User2', 'roles': ['participant']}
]
it gets similarly difficult to then find all participants of a course. Either way we have a problem to find all courses a user is subscribed to.
We think about creating a separate collection for subscriptions (back to relational):
{user_id: 'User1', course_id: '456', role: 'participant'}
{user_id: 'User1', course_id: '456', role: 'team'}
{user_id: 'User2', course_id: '456', role: 'participant'}
What is best practice?
We would like to be able to make all different sorts of queries and it seems difficult if it's buried in a nested structure...
Think this is quiet a fundamental question for data-stuctures in mongoDB.

If you don't want to change your structure I think I would do something like this using aggregation framework. In case your fields inside roles are not so many and not completely manageable by users, in that case I would suggest you to use an array in roles and unwind it.
Let's take your first structure:
Courses = [{
'name': 'Learning node.js in 1 day',
'roles': {
'team': { subscribed: [ 'User1' ] },
'participant': { subscribed: [ 'User1', 'User2' ] },
'host': { optional: true, subscribed: [] }
}
}]
Project fields in this way:
name, roles.team, roles.participant, roles.host
So you will end up with this structure
Courses = [{
'name': 'Learning node.js in 1 day',
'roles.team.subscribed': [ 'User1' ] ,
'roles.participant.subscribed': [ 'User1', 'User2' ] ,
'roles.host.subscribed': subscribed: []
}
}]
Then you can unwind subscribed field and get a cartesian product by subscribed, you need to unwind 3 times:
Courses = [
{
'name': 'Learning node.js in 1 day',
'roles.team.subscribed.values': 'User1' ,
'roles.participant.subscribed.values': 'User1',
'roles.host.subscribed.values': ''
}
},
{
'name': 'Learning node.js in 1 day',
'roles.team.subscribed.values': 'User1' ,
'roles.participant.subscribed.values': 'User2',
'roles.host.subscribed.values': ''
}
}
]
At the end you can match every role separately.
Of course you need to use aggregation framework, takes time to get use to it.

Related

How to return field based on other fields with mongoose

I have mongoose schema that looks something like this:
{
_id: someId,
name: 'mike',
keys: {
apiKey: 'fsddsfdsfdsffds',
secretKey: 'sddfsfdsfdsfdsds'
}
}
I don't want to send back to the front the keys of course, but I want some indication, like:
{
_id: someId,
name: 'mike',
hasKeys: true
}
There is built in way to create 'field' on the way based on other fields, or do I need every time fetch the whole document, check if keys is not empty and set object property based on that?
For Mongo version 4.2+ What you're looking for is called pipelined updates, it let's you use a (restricted) aggregate pipeline as your update allowing the usage of existing field values.
Here is a toy example with your data:
db.collection.updateOne(
{ _id: someId },
[
{
"$set": {
"hasKeys": {
$cond: [
{
$ifNull: [
"$keys",
false
]
},
true,
false
]
}
}
},
])
Mongo Playground
For older Mongo versions you have to do it in code.
If you don't want to update the actual document but just populate this field when you fetch it you can use the same aggregation to fetch the document
you can use $project in mongoose aggregation like this.
$project: { hasKeys: { $cond: [{ $eq: ['$keys', null] }, false, true]}}

transform raw query to mongodb query the efficient way

In a nodejs app with mongodb storage, I have the following query from user:
const rawQuery = [
'{"field":"ingredient","type":"AND","value":"green and blue"}',
'{"field":"ingredient","type":"AND","value":"black"}',
'{"field":"ingredient","type":"OR","value":"pink"}',
'{"field":"ingredient","type":"OR","value":"orange"}',
'{"field":"place","type":"AND","value":"london"}',
'{"field":"school","type":"NOT","value":"fifth"}',
'{"field":"food","type":"OR","value":"burger"}',
'{"field":"food","type":"OR","value":"pizza"}',
'{"field":"ownerFirstName","type":"AND","value":"Jimmy"}'
];
I have a collection called restaurant, and a collection called owners.
Would this query aim to handle such a search scenario?
const query = {
$and: : [
{ ingredient: 'green and blue' },
{ ingredient: 'black' },
{ $or : [
{ ingredient: 'pink' },
{ ingredient: 'orange' },
]
},
{ place: 'london' },,
{ school: { $ne: 'fifth' } },
{ $or : [
{ food: 'burger' },
{ food: 'pizza' },
]
}
]
};
How can I transform the rawQuery into this mongo query? (Given that it has to be dynamic, because I have many fields, and in this example I just included a couple of them.)
This example query aims to get the restaurants that match the description/place/school/food queries in the restaurant and also to match the owner's first name from another collection. Each restaurant document will have a ownerUuid field that points to the owner in the other collection.
What is the best solution to do a search in the mongodb for such a query in production env?
How can this be achieved with Elasticsearch?

Lowdb - push data into an array inside another array

I'm basicaly trying to do a inventory for each user, to do this i used Lowdb to help me out with the data.
My current structure is like this:
{
"users": [
{
"id": "450453034724491266",
"tag": "Briggs#4992",
"inventory": [
{"itemID": "1320488779", "rarity": "rare"},
{"itemID": "1674364779", "rarity": "common"},
]
},
{
"id": "272659147974115328",
"tag": "Zytekaron#0572",
"inventory": [
{"itemID": "1320488779", "rarity": "rare"},
{"itemID": "1674364779", "rarity": "common"},
]
}
]
}
what I have in mind is something like
//get 'users', find the id and then write inside the inventory array (which i dont know how to)
db.get('users')
.find({ id: 'id' })
Tested this code snippet and it is working perfectly.
// Get your user
let inventory = db
.get('users')
.find({ id: '272659147974115328' })
.get('inventory')
.value();
// Do something to your user's inventory
inventory.push({ itemID: '9999999', rarity: 'common' });
// Update the user with the modified inventory
db.get('users')
.find({ id: '272659147974115328' })
.assign({ inventory })
.write();
two quick answers:
get the object by the id. modify->replace-save;
you can do an assign like: object.assign({inventory:[newInventory]}).write();
where object is your user you just got from the db.
It should be possible to use push directly:
db
.get('users')
.find({ id: '272659147974115328' })
.get('inventory')
.push({ itemID: '9999999', rarity: 'common' })
.write();

How to include documents as subdocuments in MongoDB?

I have a basic user collection that consists of document like so:
{
user: 3949fj9fn9rjhfne93,
name: "Jerry",
country: 'au',
friends: ['20fn39r4hf93', 'g9en3irhr934', '4i5henifuw92']
}
Each of those friends also has a document the same, along with there being a large collection of countries that can be queried via the country code.
{
code: 'AU',
name: 'Australia'
}, {
code: 'NZ',
name: 'New Zealand'
}
My question is, how would I include the full document for each of those array items within the result, like so:
{
user: 3949fj9fn9rjhfne93,
name: "Jerry",
country: {
code: 'AU',
name: 'Australia'
},
friends: [{
user: 20fn39r4hf93,
name: "Bob",
friends: ['2049429fr', 'djwij393i4']
}, {
user: g9en3irhr934,
name: "Foo",
friends: []
}, {
user: 4i5henifuw92,
name: "Bar",
friends: ['2049429fr']
}]
}
I am using Mongoose in my application, and I understand that this could be done by using a simple for loop and pushing the results to the user object and then returning it in node with res.json(user), although what if the user had hundreds (or even thousands) of friends? The request would be huge. I also need to do this in multiple places within my API.
Is there a more efficient way to achieve this?

How define conditional "Field Selection" in mongodb 'find()' query?

Is it possible to conditionally specify the fields returned by the query. Here is my use case: I have an object with nested user conversations as follows:
{
"_id" : "someId",
user_id: 'user1',
conversations:
[
{
user_id: 'user2',
comments:
[
{
user_id: 'user2',
text: 'Hi user1'
},
{
user_id: 'user1',
text: 'Hi user2'
},
]
},
{
user_id: 'user3',
comments:
[
{
user_id: 'user3',
text: 'Hi user1'
}
]
},
]
}
I would like to allow all users to search for and view all objects but not conversations they don't own. Something as follows:
findObj = function(criteria, user, callback) {
Object.find({criteria}, {conversation:
{
if (user_id == user.id || conversations[].user_id = user.id) {1} else {0}
} }
);
}
Thanks in advance for your help,
-Eric
You can get all users that have conversations that involve user X like so :
db.users.find({$or:[{user_id: X}, {'conversations.user_id':x}]})
However this will not do what you want. You're running into a problem with your schema. You have to remove conversations from the user objects and store them in a dedicated collection that allows for queries on specific conversations seperately.

Resources