I've a JSON with next structure and data:
[ {
"id" : 716612,
"type" : "ad",
"stats" : [ {
"day" : "2020-06-01",
"impressions" : 1956,
"clicks" : 1,
"reach" : 1782
},
{
"day" : "2020-06-13",
"spent" : "73.32",
"reach" : 1059
} ]
}, {
"id" : 414290,
"type" : "campaign",
"stats" : [ {
"day" : "2020-05-21",
"effective_cost_per_click" : "31.200",
"effective_cost_per_mille" : "108.337"
},
{
"day" : "2020-05-17",
"impressions" : 1,
"reach" : 1,
"ctr" : "0.000",
"uniq_views_count" : 1
} ]
} ]
I need to map id and type from top level with data inside stats to get result like this:
[ {
"id" : 716612,
"type" : "ad",
"day" : "2020-06-01",
"impressions" : 1956,
"clicks" : 1,
"reach" : 1782
},
{
"id" : 716612,
"type" : "ad",
"day" : "2020-06-13",
"spent" : "73.32",
"reach" : 1059
},
...
I tried with:
def json = new JsonSlurper().parseText(text)
def result = json.collectMany{ a ->
a["stats"].collectMany{ b ->
b.collect{
[id: a.id,
type: a.type
]
}
}
}
But it returns only id and type fields without stats. I thought that I'm looping through stat and just adding needed fields from above. I guess I don't get the difference between collectMany and collect?
You were close 😁
You want to collect the stat plus the id and type, so you need:
def result = json.collectMany { a ->
a.stats.collect { b ->
[ id: a.id, type: a.type ] + b
}
}
Related
I have data in a collection ex:"jobs". I am trying to copy specific data from "jobs" after every 2 hours to a new collection (which may not exist initially) and also add a new key to the copied data.
I have been trying with this query to copy the data:
db.getCollection("jobs").aggregate([{ $match: { "job_name": "UploadFile", "created_datetime" : {"$gte":"2021-08-18 12:00:00"} } },{"$merge":{into: {coll : "reports"}}}])
But after this, the count in "reports" collection is 0. Also, how can I update the documents (with an extract key "report_name") without using an extra updateMany() query?
The data in jobs collection is as shown:
{
"_id" : ObjectId("60fa8e8283dc22799134dc6f"),
"job_id" : "408a5654-9a89-4c15-82b4-b0dc894b19d7",
"job_name" : "UploadFile",
"data" : {
"path" : "share://LOCALNAS/Screenshot from 2021-07-23 10-34-34.png",
"file_name" : "Screenshot from 2021-07-23 10-34-34.png",
"parent_path" : "share://LOCALNAS",
"size" : 97710,
"md5sum" : "",
"file_uid" : "c4411f10-a745-48d0-a55d-164707b7d6c2",
"version_id" : "c3dfd31a-80ba-4de0-9115-2d9b778bcf02",
"session_id" : "c4411f10-a745-48d0-a55d-164707b7d6c2",
"resource_name" : "Screenshot from 2021-07-23 10-34-34.png",
"metadata" : {
"metadata" : {
"description" : "",
"tag_ids" : [ ]
},
"category_id" : "60eed9ea33c690a0dfc89b41",
"custom_metadata" : [ ]
},
"upload_token" : "upload_token_c5043927484e",
"upload_url" : "/mnt/share_LOCALNAS",
"vfs_action_handler_id" : "91be4282a9ad5067642cdadb75278230",
"element_type" : "file"
},
"user_id" : "60f6c507d4ba6ee28aee5723",
"node_id" : "syeda",
"state" : "COMPLETED",
"priority" : 2,
"resource_name" : "Screenshot from 2021-07-23 10-34-34.png",
"group_id" : "upload_group_0babf8b7ce0b",
"status_info" : {
"progress" : 100,
"status_msg" : "Upload Completed."
},
"error_code" : "",
"error_message" : "",
"created_datetime" : ISODate("2021-07-23T15:10:18.506Z"),
"modified_datetime" : ISODate("2021-07-23T15:10:18.506Z"),
"schema_version" : "1.0.0",
}
Your $match stage contains a condition which takes created_datetime as string while in your sample data it is an ISODate. Such condtion won't return any document, try:
{
$match: {
"job_name": "UploadFile",
"created_datetime": {
"$gte": ISODate("2021-07-01T12:00:00.000Z")
}
}
}
Mongo Playground
I have a document in the below format. The goal is to group the document by student name and sort it by rank in the ascending order. Once that is done, iterate through the rank(within a student) and if each subsequent rank is greater than the previous one, the version field needs to be incremented. As part of a pipeline, student_name will be passed to me so matching by student name should be good instead of grouping.
NOTE: Tried it with python and works to some extent. A python solution would also be great!
{
"_id" : ObjectId("5d389c7907bf860f5cd11220"),
"class" : "I",
"students" : [
{
"student_name" : "AAA",
"Version" : 2,
"scores" : [
{
"value" : "50",
"rank" : 2
},
{
"value" : "70",
"rank" : 1
}
]
},
{
"student_name" : "BBB",
"Version" : 5,
"scores" : [
{
"value" : 80,
"rank" : 2
},
{
"value" : 100,
"rank" : 1
},
{
"value" : 100,
"rank" : 1
}
]
}
]
}
I tried this piece of code to sort
def version(student_name):
db.column.aggregate(
[
{"$unwind": "$students"},
{"$unwind": "$students.scores"},
{"$sort" : {"students.scores.rank" : 1}},
{"$group" : {"students.student_name}
]
)
for i in range(0,(len(students.scores)-1)):
if students.scores[i].rank < students.scores[i+1].rank:
tag.update_many(
{"$inc" : {"students.Version":1}}
)
The expected output for student AAA should be
{
"_id" : ObjectId("5d389c7907bf860f5cd11220"),
"class" : "I",
"students" : [
{
"student_name" : "AAA",
"Version" : 3, #version incremented
"scores" : [
{
"value" : "70",
"rank" : 1
},
{
"value" : "50",
"rank" : 2
}
]
}
I was able to sort the document.
pipeline = [
{"$unwind": "$properties"},
{"$unwind": "$properties.values"},
{"$sort" : {"$properties.values.rank" : -1}},
{"$group": {"_id" : "$properties.property_name", "values" : {"$push" : "$properties.values"}}}
]
import pprint
pprint.pprint(list(db.column.aggregate(pipeline)))
If I have these objects :
{
"_id" : ObjectId("5caf2c1642e3731464c2c79d"),
"requested" : [],
"roomNo" : "E0-1-09",
"capacity" : 40,
"venueType" : "LR(M)",
"seatingType" : "TB",
"slotStart" : "8:30AM",
"slotEnd" : "9:50AM",
"__v" : 0
}
/* 2 */
{
"_id" : ObjectId("5caf2deb4a7f5222305b55d5"),
"requested" : [],
"roomNo" : "E0-1-09",
"capacity" : 40,
"venueType" : "LR(M)",
"seatingType" : "TB",
"slotStart" : "10:00AM",
"slotEnd" : "11:20AM",
"__v" : 0
}
is it possible to get something like this using aggregate in mongodb?
[{ roomNo: "E0-1-09" , availability : [{slotStart : "8:30AM", slotEnd: "9:50AM"} ,
{slotStart: "10:00AM", slotEnd : "11:20AM"}]
what im using currently:
db.getDB().collection(collection).aggregate([
{ $group: {_id:{roomNo: "$roomNo", availability :[{slotStart:"$slotStart", slotEnd:"$slotEnd"}]}}}
])
actually getting it twice like so :
[{ roomNo: "E0-1-09" , availability : [{slotStart : "8:30AM", slotEnd: "9:50AM"}]
[{ roomNo: "E0-1-09" , availability : [{slotStart: "10:00AM", slotEnd : "11:20AM"}]
You have to use $push accumulator
db.collection.aggregate([
{ "$group": {
"_id": "$roomNo",
"availability": {
"$push": {
"slotEnd": "$slotEnd",
"slotStart": "$slotStart"
}
}
}}
])
I have a collection with three documents:
{ "_id" : ObjectId("5bfe572882ace71e43703d15"), "event" : "Cdr", "privilege" : "cdr,all", "accountcode" : "", "source" : "22000", "destination" : "98723546", "destinationcontext" : "from-internal", "callerid" : "\"22000\" <22000>", "channel" : "SIP/22000-00000005", "destinationchannel" : "SIP/9144502101-00000006", "lastapplication" : "Dial", "lastdata" : "SIP/9144502101/98723546,300,Tb(func-apply-sipheaders^s^1)", "starttime" : "2018-11-28 15:51:47", "answertime" : "", "endtime" : "2018-11-28 15:51:52", "duration" : "5", "billableseconds" : "0", "disposition" : "NO ANSWER", "amaflags" : "DOCUMENTATION", "uniqueid" : "1543395107.5", "userfield" : "" }
{ "_id" : ObjectId("5bfe5829b3a9321f241f10f2"), "event" : "Cdr", "privilege" : "cdr,all", "accountcode" : "", "source" : "98723546", "destination" : "s", "destinationcontext" : "ivr-1", "callerid" : "\"98723546\" <98723546>", "channel" : "SIP/9144502101-00000007", "destinationchannel" : "", "lastapplication" : "BackGround", "lastdata" : "custom/int1", "starttime" : "2018-11-28 15:56:03", "answertime" : "2018-11-28 15:56:03", "endtime" : "2018-11-28 15:56:09", "duration" : "6", "billableseconds" : "6", "disposition" : "ANSWERED", "amaflags" : "DOCUMENTATION", "uniqueid" : "1543395363.7", "userfield" : "" }
{ "_id" : ObjectId("5bfe5833b3a9321f241f10f4"), "event" : "Cdr", "privilege" : "cdr,all", "accountcode" : "", "source" : "98723546", "destination" : "22000", "destinationcontext" : "from-did-direct", "callerid" : "\"98723546\" <98723546>", "channel" : "SIP/9144502101-00000008", "destinationchannel" : "SIP/22000-00000009", "lastapplication" : "Dial", "lastdata" : "SIP/22000,,HhtrIb(func-apply-sipheaders^s^1)", "starttime" : "2018-11-28 15:56:12", "answertime" : "2018-11-28 15:56:12", "endtime" : "2018-11-28 15:56:19", "duration" : "7", "billableseconds" : "7", "disposition" : "NO ANSWER", "amaflags" : "DOCUMENTATION", "uniqueid" : "1543395372.8", "userfield" : "" }
I want to sum $duration and try command:
db.cdrs.aggregate([{$group: { _id: "$event", total: { $sum: "$duration"}}}])
Result return:
{ "_id" : "Cdr", "total" : 0 }
How to use $sum return result Sum duration of three documents(5 + 6 + 7 = 18)?
You can sum duration by converting string to Int
Its work on Mongo version 4.0
Example :
db.getCollection('A').aggregate([{
$addFields: {
convertedDuration: { $toInt: "$duration" },
} },
{
$group: { _id: "$event", total: { $sum: "$convertedDuration"} }
}
])
reference Link : link1, link2
Here is the working example of your query, click on run at the top. In your Documents, the duration is stored as String, I guess mongoDB won't convert and sum the values, please change your duration to Number while storing. Your query is fine.
Example entry:
{ "_id" : "00-01#mail.ru", " pass" : 123654, "field2" : 235689, "field3" : "cccp123654", "field4" : "lhfrjy" }
Desired result:
{ "_id" : "00-01#mail.ru", " pass" : 123654, 235689, "cccp123654", "lhfrjy" }
I want to have two final fields (_id and pass).
I have attempted the following:
db.emails.aggregate([
{ "$project": {
"pass": { "$setUnion": [ "$field2", "$field3" ] }
}}
])
However, this results in the following error:
2018-01-22T03:01:26.074+0000 E QUERY [thread1] Error: command failed: {
"ok" : 0,
"errmsg" : "All operands of $setUnion must be arrays. One argument is of type: string",
"code" : 17043,
"codeName" : "Location17043"
} : aggregate failed :
_getErrorWithCode#src/mongo/shell/utils.js:25:13
doassert#src/mongo/shell/assert.js:16:14
assert.commandWorked#src/mongo/shell/assert.js:370:5
DBCollection.prototype.aggregate#src/mongo/shell/collection.js:1319:5
#(shell):1:1
Can someone assist?
we can convert $objectToArray and $slice after 1 element in array
> db.io.aggregate(
[
{$addFields : {arr : {$objectToArray : "$$ROOT"}}},
{$project : { pass : {$slice : ["$arr.v", 1, 20 ] }}}
]
).pretty()
result
{
"_id" : "00-01#mail.ru",
"pass" : [
123654,
235689,
"cccp123654",
"lhfrjy"
]
}
>