show all 1M with sorting twice in couchdb? - couchdb

New to couchdb, I want to do a 1 to Many relationship with sorting
Category -> Documents
Would like to sort Category and sort Documents within each Category and hopefully get the result as an array.
I can sort either by Category or by Documents, but not both.
I would like to get something like this as the query result:
[{ name: 'category1',
position: 1,
type: 'category',
documents: [{ name: ..., type: ..., position: 1 },
{ name: ..., type: ..., position: 2 },
{ name: ..., type: ..., position: 3 }
}
{ name: 'category2',
position: 2,
type: 'category',
documents: [{ name: ..., type ..position: 1 },
{ name: ..., position: 2 },
{ name: ..., position: 3 }]
}]
I setup a view design and a map function like so (is this the correct approach?):
function(category) {
if (type == 'category') {
for (var d in category.documents) {
emit([category.position, category.documents[d].position??? ], {_id: category.documents[d], category: category })
}
}
Problems are...
1- category.documents[d].position wouldn't 'exist' yet so I can't simply do that.
2- the query results isn't formatted the way I would want. It would be rows of documents instead of rows of category with a array of document objects.

There are no relationships in CouchDB. The correct way would be to set the category a document belongs to directly on the document.
{
_id: ...,
name: ...,
type: ...,
category: 1,
position: 1
}

As pointed out by #OctavianDamiean, you should add a category field in the document. Then the map function becomes something like:
function(doc) {
if (doc.type === 'Document') {
emit([doc.category, doc.position], 1);
}
}
Querying with include_docs=true, you'll get:
[ { "key": ["category1", 1], "doc": { "name": "doc1.1", "type": "Document", "position": 1 } },
{ "key": ["category1", 2], "doc": { "name": "doc1.2", "type": "Document", "position": 2 } },
{ "key": ["category2", 1], "doc": { "name": "doc2.1", "type": "Document", "position": 1 } },
{ "key": ["category2", 2], "doc": { "name": "doc2.2", "type": "Document", "position": 2 } },
...
]

Related

sort array of object, then sort array of collection

here is the collection and query code. Now i want to do two things.
i) sort reportTypes array objects by counts in descending order then,
ii) sort the collection by total no. of counts in reportTypes array in descending order.
iii) then group by managerId
i want resultant doc to like this.
[
{
"_id": ObjectId("62441917d12596f96de163a3"),
"managerId": 2,
"reportTypes": [
{
"reasonId": 100,
"count": 20
}
]
},
{
"_id": ObjectId("62441917d12596f96de163a5"),
"managerId": 3,
"reportTypes": [
{
"reasonId": 200,
"count": 10
},
{
"reasonId": 100,
"count": 5
},
{
"reasonId": 300,
"count": 0
}
]
},
{
"_id": ObjectId("62441917d12596f96de163a2"),
"managerId": 1,
"reportTypes": [
{
"reasonId": 300,
"count": 4
},
{
"reasonId": 200,
"count": 3
},
{
"reasonId": 100,
"count": 2
}
]
}
]
Maybe something like this:
db.collection.aggregate([
{
$unwind: "$reportTypes"
},
{
$sort: {
"managerId": 1,
"reportTypes.count": -1
}
},
{
$group: {
_id: "$managerId",
reportTypes: {
$push: "$reportTypes"
},
cnt: {
$sum: "$reportTypes.count"
}
}
},
{
$addFields: {
managerId: "$_id"
}
},
{
$sort: {
cnt: -1
}
},
{
$project: {
managerId: 1,
reportTypes: 1
}
}
])
Explained:
Unwind the reportTypes
Sort by managerId and descending by reportTypes.count
group with push to form the same objects with sorted arrays per managerId and generate summary count per managerId.
addFileds managerId
Sort by total count ( cnt)
Project only the needed fields
playground

Mongoose/MongoDB Get mostly viewed articles grouped within a day

I am trying to clone a Reddit-like community board API using MongoDB + Mongoose on Node.js.
My sample JSON data looks like below:
{
"genre": "free",
"viewCount": 90,
"isDeleted": false,
"commentCount": 0,
"voteCount": 0,
"_comments": [],
"_vote": [],
"_id": "ObjectId",
"title": "blahblah",
"contents": "blah",
"createdAt": "2020-01-24T08:50:28.409Z",
"__v": 0,
"id": "5e2aafd4395bf593aa94b623"
},
To solve this problem, I simply sorted using .sort({ viewCount:-1, createdAt: -1 }).
However, when I sorted in this way, the most recently created Post will be always come first, even though other posts have larger viewCount values...
The next thing I'm thinking of is trying to group Posts data by each day (i.e. All posts created today is grouped together; All posts created yesterday is grouped together).
After grouping, then maybe I can sort the rest of data by viewCount.
I believe the method using aggregate would be the one possible solution, but I'd like to know if there would be the simplest and the best solution for this problem!
The output I'd like to get is something like this:
// viewcount in Descending Order
{ '2020-01-24':
{ post1: { viewcount: 999, contents: ...},
{ post2: { viewcount: 998, contents:... },
... } },
'2020-01-23':
{ post1: { viewcount: 999, contents: ...},
{ post2: { viewcount: 998, contents:... },
... },
'2020-01-22':
{ post1: { viewcount: 999, contents: ...},
{ post2: { viewcount: 998, contents:... },
... }, ...}
Please help me out here...
Thank you :)
This aggregation gives something similar to the output you are expecting:
db.test.aggregate( [
{ $sort: { createdAt: -1, viewCount: -1} },
{ $group: { _id: "$createdAt", post: { $push: "$$ROOT" } } },
{ $project: { post: 1, date: "$_id", _id: 0 } }
] )

Mongodb - populate with limit on items and get total count of those items

I have a query looking like this:
const articles = await Article.find(query)
.populate({
path: 'votedUsers', // array of users id
select: 'title name username',
options: {
limit: 3,
sort: { createdAt: -1 },
},
})
.exec()
Result:
[
{
title: 'Article title',
votedUsers: [,,], // array of populated users with limit of 3
totalCountVoted: 200 // need to add this field
}
]
I want to find articles and populate votedUsers property but with limit to 3 users, but at the same time
I need to know how many ids were in votedUsers property.
For example it can be 200 users that voted on that article, but I just need to know the number and populate only 3 of them.
You can try the following aggregation using the match, lookup, project stages, and slice and size operators:
(Please note that the "users" value in lookup from must be the physical collection name.)
app.get("/article", async (req, res) => {
const data = await Article.aggregate([
{
$match: {
category: "Category1"
}
},
{
$lookup: {
from: "users",
localField: "votedUsers",
foreignField: "_id",
as: "users"
}
},
{
$project: {
title: 1,
votedUsers: { $slice: ["$users", 3] },
totalCountVoted: { $size: "$users" }
}
}
]);
res.send(data);
});
This will give you a result like this:
[
{
"_id": "5dded78f8f30c402b0fac309",
"title": "Article1",
"votedUsers": [
{
"_id": "5dded60a84523642bc27f511",
"__v": 0,
"name": "User1"
},
{
"_id": "5dded61384523642bc27f512",
"__v": 0,
"name": "User2"
},
{
"_id": "5dded61b84523642bc27f513",
"__v": 0,
"name": "User3"
}
],
"totalCountVoted": 8
},
{
"_id": "5dded7c18f30c402b0fac30a",
"title": "Article2",
"votedUsers": [
{
"_id": "5dded61b84523642bc27f513",
"__v": 0,
"name": "User3"
},
{
"_id": "5dded63c84523642bc27f514",
"__v": 0,
"name": "User4"
},
{
"_id": "5dded64484523642bc27f515",
"__v": 0,
"name": "User5"
}
],
"totalCountVoted": 8
}
]
Playground

Limiting number of different kind of results in mongoDB aggregation

I would like to write an aggregation that take the following documents:
{ type: "dog", name: "Charlie" }
{ type: "dog", name: "Felix" }
{ type: "dog", name: "John" }
{ type: "cat", name: "Tum" }
And returns up to 2 of each kind, not grouped in any separate way:
{ type: "dog", name: "Charlie" }
{ type: "dog", name: "Felix" }
{ type: "cat", name: "Tum" }
Meaning just up to two cats plus up to two dogs. Does grouping and limiting the way to go here? If so - how?
You can group the documents by type, create a list of the names per group, $project the list to only have two elements with $slice and then flatten the list using $unwind, something like the following:
Model.aggregate([
{ "$group": {
"_id": "$type",
"data": { "$push": "$name" }
} },
{ "$project": {
"type": "$_id",
"name": { "$slice": ["$data", 2] }
} },
{ "$unwind": "$name" }
]).exec(callback);

Return records in mongoDB based on top 1 of field x grouped by type y

If i have the following json structure:
[
{
id: 1,
type: "Drinks",
isActive : "true",
location: "QLD"
},
{
id: 2,
type: "Drinks",
isActive : "false",
location: "NSW"
},
{
id: 3,
type: "Drinks",
isActive : "true"
location: "QLD"
},
{
id: 3,
type: "Drinks",
isActive : "false"
location: "QLD"
},
{
id: 3,
type: "Drinks",
isActive : "true"
location: "QLD"
},
{
id: 4,
type: "Food",
isActive : "true"
location: "NSW"
},
{
id: 4,
type: "Food",
isActive : "false"
location: "NSW"
}
]
The return i'm interested in is:
[
{
id: 1,
type: "Drinks",
isActive : "true",
location: "QLD"
},
{
id: 2,
type: "Drinks",
isActive : "false",
location: "NSW"
},
{
id: 3,
type: "Drinks",
isActive : "true",
location: "QLD"
},
{
id: 4,
type: "Food",
isActive : "false",
location: "NSW"
}
]
In other words, give me top 1 of each TYPE in each LOCATION sorted by ID descending. The records may be repeated as the sample data set looks so essentially i want all the unique types for each location. Is this something that can be done in mongoD?
It many not be relevant but i am using Mongoose within my nodejs app to interact with mongoDB.
The aggregation framework is at your disposal to give you the desired results. You would have to run an aggregation pipeline consists of 3 stages, in the following order:
$sort
This pipe will allow you to first order the documents getting into the pipeline for grouping later. Sort the documents with the ID descending.
$group
The group pipeline operator is similar to the SQL's GROUP BY clause. In SQL, you can't use GROUP BY unless you use any of the aggregation functions. The same way, you have to use an aggregation function in MongoDB as well.
In this instance, you need to group all the documents by the type, location and id keys, and use the required $first operator to bring in the first document (in other words, the TOP document when ordered).
$project
This pipeline step is similar to SELECT in SQL. Use this to rename the field names and select/deselect the fields to be returned, out of the grouped fields. If you specify 0 for a field, it will NOT be sent in the pipeline to the next operator.
Putting all the three pipes together, you can run the following aggregation pipeline to achieve the required result:
var pipeline = [
{ "$sort": { "id": -1 } },
{
"$group": {
"_id": {
"type": "$type", "location": "$location", "id": "$id"
},
"isActive": { "$first": "$isActive" }
}
},
{
"$project": {
"_id": 0,
"id": "$_id.id",
"isActive": 1,
"type": "$_id.type",
"location": "$_id.location"
}
}
]
Model.aggregate(pipeline, function(err, result) {
if err throw err;
console.log(result);
});
Or using the fluent API
Model.aggregate()
.sort("-id")
.group({
"_id": {
"type": "$type", "location": "$location", "id": "$id"
},
"isActive": { "$first": "$isActive" }
})
.project({
"_id": 0,
"id": "$_id.id",
"isActive": 1,
"type": "$_id.type",
"location": "$_id.location"
})
.exec(unction(err, result) {
if err throw err;
console.log(result);
});
Try the following query:-
db.collname.aggregate(
[
{
$group:
{
_id: "$type",
id: { $max: "$id" }
}
}
]
Refer doc for more info.

Resources