I have a collection with multiple documents with this format:
{
"id" : "65451735",
"recommendations" : [
{
"lastModifiedDate" : ISODate("2018-12-07T00:00:00.000Z"),
"products" : [
{
"productName" : "name1",
"productId" : "productid1"
},
{
"productName" : "name2",
"productId" : "productid1"
}
],
"recommendationsValueGlobal" : 0.0
}
]
}
I have with filter with "id" and "productId".
There's 2 issues regarding the productId:
1.- it can accept comma separated values of multiple product ids
2.- the productId is really deeply nested, I've never searched through an object this deep in the document. There can be multiple recommendations, and multiple products
I need to return all documents where id matches and only include any of the products that have the Ids given, all other should not be displayed.
I know how to search on nested object inside an array, but this is way too much, specially with comma separated values. Any suggestion?
EDIT: Also the Id is not unique and can be repeated in multiple documents
Related
I need to check the amount of occurrence for a specific product id in a high complexity array of objects structure.
I tried with the aggregation functionality of MongoDB, but I only got the occurrences per document rather than the occurrences cross all the documents. The data structure of each document looks like this:
{
"_id" : ObjectId("5c4d67905a07f5dec1fcc763"),
"updatedAt" : ISODate("2019-01-27T08:16:11.706Z"),
"createdAt" : ISODate("2019-01-27T08:10:56.553Z"),
"pickupTime" : ISODate("2019-01-27T08:20:00.000Z"),
"shop" : ObjectId("5c24b007b55ea3d7c599c95b"),
"owner" : ObjectId("5c242e361ee775cdd8b047b6"),
"total" : 350,
"status" : "completed",
"lineItems" : [
{
"product" : {
"options" : [],
"enabled" : false,
"description" : "",
"shop" : ObjectId("5c24b007b55ea3d7c599c95b"),
"price" : 350,
"name" : "Capuccino",
"createdAt" : ISODate("2019-01-01T21:56:31.928Z"),
"updatedAt" : ISODate("2019-01-08T12:14:53.322Z"),
"_id" : ObjectId("5c2be20f8e52115849726fdc")
},
"_id" : ObjectId("5c48efbc9efde32fab7ae47d"),
"status" : "pending",
"quantity" : 1,
"config" : []
}
],
"__v" : 0
}
As you can see each document is a order, each order has lineItems that is a array of objects that contain the quantity of the desired product, the status of the current product, some configuration and the product itself. The product field is a snapshot of the product in the moment of the order as they may be changed by the supplier (cost, name description, etc.)
I currently tried the next aggregate formula:
{
$group: {
_id: {
'product': '$lineItems.product._id'
},
count: {
$sum: 1
}
}
}
But only returns the the occurrences of a product in each order, rather than cross all the orders. I understood I need to use $reduce to reformat the data structure, but I couldn't find a way how to organice my current data structure in order to make it match. Also I got said that after counting per product I can use the $sort to put the highest values on the top and then limit the search to the amount of records to return
The desirable result is get the top 5 most sold products based on the orders data.
Thanks.
I have a collection data which has around 300k entries and its document looks like
{
"_id" : ObjectId("5xxx85"),
"user_id" : "1",
"name" : "test",
"user_private" : "0"
}
now i want to update all the documents in this collection and new document will look like
{
"_id" : ObjectId("5xxx85"),
"rid" : "1",
"user_name" : "test",
"is_private" : "private",
"is_moderator" : "true",
"amount" : "11111"
}
i.e i need to add new fields, update field names and check if user_private = 0 then put is_private as private or else put is_private as not_private.
I am a bit new so I am not able to get how can i do this efficiently as entries are around 300k.
Please suggest some ways, pseudo code will be really helpful
To update a document a filter criteria. Check pseudo code below and follow link to read more.
You'll need to have an existing value for user_private
db.messages.updateMany([
{ "user_private" : 0 }, // filter.
{ $set: {
"user_private" : "private",
"is_moderator" : "true"
}
}, // update.
{
upsert: true
}
]);
upserts - Creates a new document if no documents match the filter or Updates documents that match the filter based on the filter and update parameters provided.
I'm using MongoDb (as part of MongoJS) in Node. I using subdocuments and allocating IDs to the sub-docs also.
when I do a query based on the main documents it returns me the whole document, but when I try to find the sub-document based on its ID it do't return me any result. In mongoose I have declared a seperate schema for results field having its own ID.
It is to note that I am using arrays within arrays, for the result field.
Following is the console output of this scenario.
>db.tests.find({"_id":ObjectId("56563e92c8be03ec1a341374")}).pretty();
"_id" : ObjectId("56563e92c8be03ec1a341374"),
"test" : 2,
"startAt" : ISODate("2015-11-25T23:04:50Z"),
"endedAt" : ISODate("2015-11-25T23:04:50Z"),
"results" : [
{
"_id" : ObjectId("56563e92c8be03ec1a341375"),
"second" : [
{
"sec" : 50,
"avg" : 40.6,
"grt" : 1.2
}
]
}
],
>db.tests.find({"_id":ObjectId("56563e92c8be03ec1a341375")}).pretty();
>
You cannot search for subdocuments only by their ID in mongoDb. You have to always start from the main document i.e...search for the document that has an "results" array that contains the subdocument you are looking for:
db.test.find({ "results._id" : ObjectId("56563e92c8be03ec1a341375") });
if you want the output document to contain only the subdocument you were looking for (excluding other subdocuments it the array) you can use projection {"results.$":1}:
db.test.find({ "results._id" : ObjectId("56563e92c8be03ec1a341375") },{"results.$" : 1});
It will give you this output:
{
"_id": ObjectId("56563e92c8be03ec1a341374"),
"results": [{
"_id": ObjectId("56563e92c8be03ec1a341375"),
"second": [{
"sec": 50,
"avg": 40.6,
"grt": 1.2
}]
}]
}
So I have this document within the course collection
{
"_id" : ObjectId("53580ff62e868947708073a9"),
"startDate" : ISODate("2014-04-23T19:08:32.401Z"),
"scoreId" : ObjectId("531f28fd495c533e5eaeb00b"),
"rewardId" : null,
"type" : "certificationCourse",
"description" : "This is a description",
"name" : "testingAutoSteps1",
"authorId" : ObjectId("532a121e518cf5402d5dc276"),
"steps" : [
{
"name" : "This is a step",
"description" : "This is a description",
"action" : "submitCategory",
"value" : "532368bc2ab8b9182716f339",
"statusId" : ObjectId("5357e26be86f746b68482c8a"),
"_id" : ObjectId("53580ff62e868947708073ac"),
"required" : true,
"quantity" : 1,
"userId" : [
ObjectId("53554b56e3a1e1dc17db903f")
]
},...
And I want to do is create a query that returns all courses that have a specific userId in the userId array that is in the steps array for a specific userId. I've tried using $elemMatch like so
Course.find({
"steps": {
"$elemMatch": {
"userId": {
"$elemMatch": "53554b56e3a1e1dc17db903f"
}
}
}
},
But It seems to be returning a empty document.
I think this will work for you, you have the syntax off a bit plus you need to use ObjectId():
db.Course.find({ steps : { $elemMatch: { userId:ObjectId("53554b56e3a1e1dc17db903f")} } })
The $elemMatch usage is not necessary unless you actually have compound sub-documents in that nested array element. And also is not necessary unless the value being referenced could possibly duplicate in another compound document.
Since this is an ObjectId we are talking about, then it's going to be unique, at least within this array. So just use the "dot-notation" form:
Course.find({
"steps.userId": ObjectId("53554b56e3a1e1dc17db903f")
},
Go back and look at the $elemMatch documentation. In this case, the direct "dot-notation" form is all you need
I am searching within a collection of Stores. Stores have an embedded collection of outlets with locations. My goal is to return the set of stores that have outlets near a geolocation, and also only return those Outlets within that location.
I can successfully restrict the query to only return Stores have an Outlet at a particular location using 'near'
Store
.where('isActive').equals(true)
.where('outlets.location')
.near({ center: [153.027117, -27.468515], maxDistance: 1000 / 6378137, spherical: true })
.where('outlets.isActive').equals(true)
.where('products.productType').equals('53433f1f3e02e39addde1954')
.where('products.isActive').equals(true)
.select('name outlets')
.select({'products': {$elemMatch: {'isActive': true, 'productType': '53433f1f3e02e39addde1954'}}})
.select('name outlets')
.execQ()
.then(function (results) {
console.log(results);
})
.fail(function (err) {
console.log(err);
})
.done();
The problem I have is that the store document returns all the outlets, not just the outlet that matched the geolocation. I've tried using elemMatch within a select like I did with the products;
.select({'outlets': {$elemMatch: {'location': {near:{ center: [153.027117, -27.468515], maxDistance: 10000 / 6378137, spherical: true }}}}})
However it returns an empty array. Can use use the near operator in an elemMatch clause? Am I doing it incorrectly? Is there an more efficient/fast/better way to achieve the goal?
I see what you are trying to do here but there seems to be a few flaws in this sort of design. Though not exactly your document structure I see you are trying to do something like this:
{
"_id" : ObjectId("5344badd519563414f23fdf8"),
"store" : "Mine",
"outlets" : [
{
"name" : "somewhere",
"loc" : {
"type" : "Point",
"coordinates" : [
150.975131,
-33.8440366
]
}
},
{
"name" : "else",
"loc" : {
"type" : "Point",
"coordinates" : [
151.3651524,
-33.8389783
]
}
}
]
}
{
"_id" : ObjectId("5344be6f519563414f23fdf9"),
"store" : "Another",
"outlets" : [
{
"name" : "else",
"loc" : {
"type" : "Point",
"coordinates" : [
151.3651524,
-33.8389783
]
}
},
{
"name" : "somewhere",
"loc" : {
"type" : "Point",
"coordinates" : [
150.975131,
-33.8440366
]
}
}
]
}
So basically you appear to be attempting to nest the outlet locations within an array in a top level document.
What I am referring to a flaw here is that by design, any type of "near" based query is going to return more than 1 result. That does seem logical when you look at the purpose. You can of course modify this to restrict the results by "maxDistance" but generally it will be more than 1.
So the only way is to .limit() the results returned by the cursor to a single "nearest" response. Also note that with some operations those results are not necessarily "sorted" with the "nearest response first.
Now as these results are actually contained within an array of the document, remember that .find() itself does not actually "filter" the results of an array, so of course the document will contain all of the array contents.
If you tried to "project" with a positional $ operator, then the problem falls back to the original point because there is no singular actual match, so it is not possible to return an "index" value for the matching element. If you in fact did try this, you would always get the default index value of 0, so just returning the first element.
If then you thought you could run off to aggregate and and try to actually "de-normalize" the array entries, you would be out of luck because due to the need to use the index at the first stage of any aggregation pipeline statement.
So the summary of this is that embedded entries like this are not suited to this design where you need to do geo-spatial matching on those store locations. The locations are better off in a separate collection:
{
"_id" : ObjectId("5344bec7519563414f23fdfa"),
"store": "Mine"
"name" : "else",
"loc" : {
"type" : "Point",
"coordinates" : [
151.3651524,
-33.8389783
]
}
}
{
"_id" : ObjectId("5344bed5519563414f23fdfb"),
"store": "Mine"
"name" : "somewhere",
"loc" : {
"type" : "Point",
"coordinates" : [
150.975131,
-33.8440366
]
}
}
So that would allow you to "limit" the result to the "nearest" match by setting the limit to 1. You can also include any necessary values such as the "store" to be used in your filtering. If you need to you can include other information aside from what you need to filter or otherwise just put the ObjectId values within the array of the original object, or possibly even duplicate for both collections.
But since the very nature of these queries is intended to not only return 1 match, then there is no way you are going to get this to work on embedded documents. So your solution will require some changes in your overall schema.