This is my Friends Collection
[
{
"_id": "59e4fbcac23f38cdfa6963a8",
"friend_id": "59e48f0af8c277d7a8886ed7",
"user_id": "59e1d36ad17ad5ad3d0453f7",
"__v": 0,
"created_at": "2017-10-16T18:34:50.875Z"
},
{
"_id": "59e5065f705a90cfa218c9e5",
"friend_id": "59e48f0af8c277d7a8886edd",
"user_id": "59e1d36ad17ad5ad3d0453f7",
"__v": 0,
"created_at": "2017-10-16T19:19:59.483Z"
}
]
This is my Scores collection:
[
{
"_id": "59e48f0af8c277d7a8886ed8",
"score": 19,
"user_id": "59e48f0af8c277d7a8886ed7",
"created_at": "2017-10-13T09:02:10.010Z"
},
{
"_id": "59e48f0af8c277d7a8886ed9",
"score": 24,
"user_id": "59e48f0af8c277d7a8886ed7",
"created_at": "2017-10-11T00:56:10.010Z"
},
{
"_id": "59e48f0af8c277d7a8886eda",
"score": 52,
"user_id": "59e48f0af8c277d7a8886ed7",
"created_at": "2017-10-24T09:16:10.010Z"
},
]
This is my Users collection.
[
{
"_id": "59e48f0af8c277d7a8886ed7",
"name": "testuser_0",
"thumbnail": "path_0"
},
{
"_id": "59e48f0af8c277d7a8886edd",
"name": "testuser_1",
"thumbnail": "path_1"
},
{
"_id": "59e48f0af8c277d7a8886ee3",
"name": "testuser_2",
"thumbnail": "path_2"
},
{
"_id": "59e48f0af8c277d7a8886ee9",
"name": "testuser_3",
"thumbnail": "path_3"
},
]
And finally i need list of friends sorted in highscore order for a particular time period (say last 24 hours) with something like this...
[
{
"friend_id": "59e48f0af8c277d7a8886ed7",
"friend_name":"test_user_2"
"thumbnail":"image_path",
"highscore":15
},
"friend_id": "59e48f0af8c277d7a8886edd",
"friend_name":"test_user_3"
"thumbnail":"image_path",
"highscore":10
}
]
What's the best way to achieve this? I have tried aggregation pipeline but getting quite confused with working with 3 collections.
Following your answers, an array size of 500 entries in a document may not be a bad idea to store the friends as you would only store "friends id" and "created" in each entry. It saves having a collection.
You would not have too much performances issues if you project the data in your query by selecting only the fields you want.
https://docs.mongodb.com/v3.2/tutorial/project-fields-from-query-results/#return-specified-fields-only
For the score that increase of 30 per day; it depends what type of query you do.
It would take a while to reach the 2MB limit per the document by adding 30 scores per day.
regarding joining the different collections there is a stack overflow question about it:
How do I perform the SQL Join equivalent in MongoDB?
or
https://docs.mongodb.com/manual/reference/operator/aggregation/lookup/
You will need to use the aggregation framework from mongoDB to use if; not just a find command.
Related
I'm working with a dataset similar to ArangoDB official "friendship" example, except I'm adding a "weight" concept on the Edge Collection. Like so :
People
[
{ "_id": "people/100", "_key": "100", "name": "John" },
{ "_id": "people/101", "_key": "101", "name": "Fred" },
{ "_id": "people/102", "_key": "102", "name": "Jacob" },
{ "_id": "people/103", "_key": "103", "name": "Ethan" }
]
Friendship
[
{ "_from": "people/100", "_to": "people/101", "weight": 27 },
{ "_from": "people/103", "_to": "people/102", "weight": 31 },
{ "_from": "people/102", "_to": "people/100", "weight": 12 },
{ "_from": "people/101", "_to": "people/103", "weight": 56 }
]
I want to write a function that, when someone interacts with someone else, UPSERTs the Friendship between the two (incrementing the weight by 1 if it existed before, or initializing with a weight of 1 if it's new).
The trouble is, when executing that function, I have now clue on which direction the friendship was initialized, thus I cannot really use an upsert. So 2 questions here :
Is there any way to make an upsert on an edge with "bidirectional" filter ?
Like so, but bidirectional
UPSERT {
// HERE, I BASICALLY WAN'T TO IGNORE THE SIDE
_from: ${people1}, _to: ${people2}
}
INSERT {
_from: ${people1}, _to: ${people2}, weight: 1
}
UPDATE {
weight: OLD.weight + 1
}
IN ${friendshipCollection}
RETURN NEW
Instead of trying to "select the friendship, no matter the direction"; should I rather actually duplicate the friendship on both directions (and constantly maintain / update it) ?
{
"items": [
{
"id": "5bb619e49593e5d3cbaa0b52",
"name": "Flowers",
"weight": "1.5"
},
{
"id": "5bb619e4ebdccb9218aa9dcb",
"name": "Chair",
"weight": "8.4"
},
{
"id": "5bb619e4911037797edae511",
"name": "TV",
"weight": "20.8"
},
{
"id": "5bb619e4504f248e1be543d3",
"name": "Skateboard",
"weight": "5.9"
},
{
"id": "5bb619e40fee29e3aaf09759",
"name": "Donald Trump statue",
"weight": "18.4"
},
{
"id": "5bb619e44251009d72e458b9",
"name": "Molkkÿ game",
"weight": "17.9"
},
{
"id": "5bb619e439d3e99e2e25848d",
"name": "Helmet",
"weight": "22.7"
}
]
}
I have this structure of models. I want to calculate the weight of each order.
Should I use aggregation or does someone have any idea?
this is an example of order :
{
"id": "5bb61dfd4d64747dd8d7d6cf",
"date": "Sat Aug 11 2018 02:01:25 GMT+0000 (UTC)",
"items": [
{
"item_id": "5bb619e44251009d72e458b9",
"quantity": 4
},
{
"item_id": "5bb619e4504f248e1be543d3",
"quantity": 2
},
{
"item_id": "5bb619e40fee29e3aaf09759",
"quantity": 3
}
]
}
You can use below aggregation
db.order.aggregate([
{ "$unwind": "$items" },
{ "$lookup": {
"from": "items",
"localField": "items.item_id",
"foreignField": "id",
"as": "item"
}},
{ "$unwind": "$item" },
{ "$addFields": { "items.weight": "$item.weight" }},
{ "$group": {
"_id": "$_id",
"items": { "$push": "$items" },
"date": { "$first": "$date" }
}}
])
You have two options here without changing your model structure:
pull all items used in Parcel from database in your application
perform all computations on database side using aggregation (and $lookup)
It very depends on your actual data model and dataset size. First option is very straightforward and potentially can be more performant on big datasets especially when sharding/replica set involved. But it requires more roundtrips to database which will bring more latency. On the other hand aggregation in certain cases can be quite slow on lookups.
But the only good way is to test it on your real data. If your current dataset is tiny (say 100s of Mb) choose the way you comfortable with - both will work great.
Update
Since you need to distribute Orders to Parcels I'd prefer to go with option #1, though using aggregation is still possible.
This is what I would do:
pull an Order from database
pull all related Items from database by ids found in Order.items
perform calculation of Order weight
create one Parcel if weight < 30 and save it to database
or if weight > 30 distribute somehow Items to Parcels and save them to database
Note, that you can pull multiple Items by their ids in one call with query like this:
{
_id: { $in: [<id1>, <id2>] }
}
There is also one more thing to consider. Please pay attention to the fact that MongoDB do not have transactions or multidocument atomicity. So performing this type of operations (pulling something from DB, performing calculations, and storing back) with schema defined the way you show can lead to creating duplicates.
I am trying to get data using elastic search in a python program. Currently I am getting the following data from an elastic search request. I wish to sort the data on rank:type. For example i want to sort data by raw_freq or maybe by score.
What should the query look like?
I believe it will be something using nested query. Help would be very much appreciated.
{
"data": [
{
"customer_id": 108,
"id": "Qrkz-2QBigkG_fmtME8z",
"rank": [
{
"type": "raw_freq",
"value": 2
},
{
"type": "score",
"value": 3
},
{
"type": "pmiii",
"value": 1.584962
}
],
"status": "pending",
"value": "testingFreq2"
},
],
}
Here is a simple example of how you can sort your data:
"query": {
"term": {"status": "pending"}
},
"sort": [
{"rank.type.keyword": {"order" : "desc"}}
]
A user has many payments, a payment has many debtors, a debtor belongs to a user.
I am trying to find a users payments that relate to another particular user.
I have a query which gets a users payments, populated with all the debtors and user information for each debtor.
const user_1 = await this.userModel
.findOne({email: "geoffery.brown#gmail.com"})
.populate({path: 'payments', populate: {path: 'debtors', populate: {path: 'user'}}})
which returns a something like this:
{
"payments": [
{
"debtors": [
{
"_id": "5a9531b0de918e42c94947cc",
"amount": 15,
"user": {
"payments": [],
"created_at": "2018-02-27T10:14:39.847Z",
"_id": "5a95300388740142774f49c9",
"first_name": "John",
"last_name": "Smith",
"email": "john.smith#gmail.com",
"__v": 0
},
"__v": 0
},
{
"_id": "5a9531b0de918e42c94947cd",
"amount": 10,
"user": {
"payments": [],
"created_at": "2018-02-27T10:14:39.847Z",
"_id": "5a95302188740142774f49ca",
"first_name": "Joe",
"last_name": "Blogs",
"email": "joe.blogs#hotmail.com",
"__v": 0
},
"__v": 0
}
],
"created_at": "2018-02-27T10:23:31.561Z",
"_id": "5a9531b0de918e42c94947ce",
"date": "2018-02-26T10:54:36.167Z",
"reference": "Food",
"__v": 0
}
],
"created_at": "2018-02-27T10:14:39.847Z",
"_id": "5a952fc488740142774f49c8",
"first_name": "Geoffery",
"last_name": "Brown",
"email": "geoffery.brown#gmail.com",
"__v": 0
}
I want to have my mongo query be able to filter the debtors where email === "john.smith#gmail.com"
Is this possible with my current mongodb structure?
No such support is available in mongoose populate. You can use aggregate like below in 3.4.
Similar concept as to populate but all the heavy lifting is done in single server call inside aggregation framework.
$lookup stage is used to pull data from different referenced collections.
$unwind stage to flatten the structure for subsequent lookups.
$group stages each to push debtors into payments and payments array into main document.
this.userModel.aggregate([
{"$match":{"email": "geoffery.brown#gmail.com"}},
{"$lookup":{
"from":"payments", // name of the collection
"localField":"payments",
"foreignField":"_id",
"as":"payments"
}},
{"$unwind":"$payments"},
{"$lookup":{
"from":"debtors", // name of the collection
"localField":"payments.debtors",
"foreignField":"_id",
"as":"debtors"
}},
{"$project":{"payments.debtors":0}},
{"$unwind":"$debtors"},
{"$lookup":{
"from":"users", // name of the collection
"localField":"debtors.user",
"foreignField":"_id",
"as":"debtors.user"
}},
{"$unwind":"$debtors.user"},
{"$match":{"debtors.user.email":"john.smith#gmail.com"}},
{"$group":{
"_id":{id:"$_id",payment_id:"$payments._id"},
"created_at":{"$first":"$created_at"},
"first_name":{"$first":"$first_name"},
"last_name": {"$first":"$last_name"},
"email": {"$first":"$email"},
"payments":{"$first":"$payments"},
"debtors":{"$push":"$debtors"}
}},
{"$addFields":{"payments.debtors":"$debtors"}},
{"$project":{"debtors":0}},
{"$group":{
"_id":"$_id.id",
"created_at":{"$first":"$created_at"},
"first_name":{"$first":"$first_name"},
"last_name": {"$first":"$last_name"},
"email": {"$first":"$email"},
"payments":{"$push":"$payments"}
}}
]).exec(function() {...})
This question already has answers here:
$lookup on ObjectId's in an array
(7 answers)
MongoDb aggregation $lookup with foreign _ids in arrays
(1 answer)
Closed 5 years ago.
I am a starter with mean stack, i want to nested object with another collection in mongoose, my product collection look as follows, it contain multiple bidders(users)
{
"_id": "5a16b3774d74d10ce0131254",
"name": "my test product",
"desc": "looking good",
"bid_amount": 120,
"bidders": [
{
"amount": "145",
"bid_status": "participated",
"user_id": "5a1b9f8c356e9d246c0443ac"
},
{
"amount": "160",
"bid_status": "rejected",
"user_id": "5a1be6c7356e9d246c0443af"
}
]
}
my user collection as follows
[
{
"_id": "5a1b9f8c356e9d246c0443ac",
"name": "abcd",
"email": "abcd#gmail.com"
},
{
"_id": "5a1be6c7356e9d246c0443af",
"name": "xyz",
"email": "xyz#gmail.com"
},
]
I need each user details corresponds to its object, not a separate object array, I am looking a result like this using mongoose
[
{
"_id": "5a16b3774d74d10ce0131254",
"name": "my test product",
"desc": "looking good",
"bid_amount": 120,
"bidders": [
{
"amount": "145",
"bid_status": "participated",
"user_id": "5a1b9f8c356e9d246c0443ac",
"user_details" : {
"_id": "5a1b9f8c356e9d246c0443ac",
"name": "abcd",
"email": "abcd#gmail.com"
}
},
{
"amount": "160",
"bid_status": "rejected",
"user_id": "5a1be6c7356e9d246c0443af",
"user_details": {
"_id": "5a1be6c7356e9d246c0443af",
"name": "xyz",
"email": "xyz#gmail.com"
}
}
}
]