In my nodejs app I have collection - football_players. Docs in this collection have information about role of the player. For example:
[
{
id: 'some_id_1',
name: 'Player1',
role: 'FORWARD',
},
{
id: 'some_id_2',
name: 'Player2',
role: 'DEFENDER',
},
{
id: 'some_id_3',
name: 'Player3',
role: 'FORWARD',
},
]
All types of role are constants. What is the best way to store it in MongoDB:
as strings (like in my example)
make declaration:
const roles = { FORWARD: 1, DEFENDER: 2 };
store as numbers and use this declaration in my nodejs app? Like: player.role === roles.FORWARD ? 'Great!' : 'He cann\'t score a goal'; ?
Is there any performance reason to use second way? Are there any other reasons to use first way?
Related
I have a Person object as follows
{
name: 'John Doe'
properties:
[
{
name: 'eyeColor',
value: 'brown'
},
{
name: 'age',
value: 25
},
{
name: 'interest',
value: 'reading'
},
{
name: 'interest',
value: 'diving'
}
]
}
Now I want to be able to filter my object based on multiple properties. In pseudocode:
Return all people for which
there exists any p.property such that
p.property.propertyname == 'interest'
AND p.property.propertyvalue == 'reading'
AND there exists any p.property such that
p.property.propertyname == 'age'
AND p.property.propertyvalue < 30
What is the most concise and extensible (I want to be able to apply N of these filters) of doing this without having too many intermediate results?
Imaging I have an array of objects, available before the aggregate query:
const groupBy = [
{
realm: 1,
latest_timestamp: 1318874398, //Date.now() values, usually different to each other
item_id: 1234, //always the same
},
{
realm: 2,
latest_timestamp: 1312467986, //actually it's $max timestamp field from the collection
item_id: 1234,
},
{
realm: ..., //there are many of them
latest_timestamp: ...,
item_id: 1234,
},
{
realm: 10,
latest_timestamp: 1318874398, //but sometimes then can be the same
item_id: 1234,
},
]
And collection (example set available on MongoPlayground) with the following schema:
{
realm: Number,
timestamp: Number,
item_id: Number,
field: Number, //any other useless fields in this case
}
My problem is, how to $group the values from the collection via the aggregation framework by using the already available set of data (from groupBy) ?
What have been tried already.
Okay, let skip crap ideas, like:
for (const element of groupBy) {
//array of `find` queries
}
My current working aggregation query is something like that:
//first stage
{
$match: {
"item": 1234
"realm" [1,2,3,4...,10]
}
},
{
$group: {
_id: {
realm: '$realm',
},
latest_timestamp: {
$max: '$timestamp',
},
data: {
$push: '$$ROOT',
},
},
},
{
$unwind: '$data',
},
{
$addFields: {
'data.latest_timestamp': {
$cond: {
if: {
$eq: ['$data.timestamp', '$latest_timestamp'],
},
then: '$latest_timestamp',
else: '$$REMOVE',
},
},
},
},
{
$replaceRoot: {
newRoot: '$data',
},
},
//At last, after this stages I can do useful job
but I found it a bit obsolete, and I already heard that using [.mapReduce][1] could solve my problem a bit faster, than this query. (But official docs doesn't sound promising about it) Does it true?
As for now, I am using 4 or 5 stages, before start working with useful (for me) documents.
Recent update:
I have checked the $facet stage and I found it curious for this certain case. Probably it will help me out.
For what it's worth:
After receiving documents after the necessary stages I am building a representative cluster chart, that you may also know as a heatmap
After that I was iterating each document (or array of objects) one-by-one to find their correct x and y coordinated in place which should be:
[
{
x: x (number, actual $price),
y: y (number, actual $realm),
value: price * quantity,
quantity: sum_of_quantity_on_price_level
}
]
As for now, it's old awful code with for...loop inside each other, but in the future, I will be using $facet => $bucket operators for that kind of job.
So, I have found an answer to my question in another, but relevant way.
I was thinking about using $facet operator and to be honest, it's still an option, but using it, as below is a bad practice.
//building $facet query before aggregation
const ObjectQuery = {}
for (const realm of realms) {
Object.assign(ObjectQuery, { `${realm.name}` : [ ... ] }
}
//mongoose query here
aggregation([{
$facet: ObjectQuery
},
...
])
So, I have chosen a $project stage and $switch operator to filter results, such as $groups do.
Also, using MapReduce could also solve this problem, but for some reason, the official Mongo docs recommends to avoid using it, and choose aggregation: $group and $merge operators instead.
What is the best way to replace specific values using a Azure Data factory?
The case, for example: Need to lead to a single value brand "ssang yong" and model "ceed" for brand "kia".
Data source:
{
id: 1
brand: "ssang yong",
model: "rexton"
},
{
id: 2
brand: "ssang_yong",
model: "rexton"
},
{
id: 3
brand: "ssangyong",
model: "rexton"
},
{
id: 4
brand: "kia",
model: "ceed"
},
{
id: 5
brand: "kia",
model: "c'eed"
}
Pattern:
{
target: "brand",
common_value: "ssang yong",
condition: {
brand: ["ssang-yong", "ssangyong"]
}
},
{
target: "model",
common_value: "ceed",
condition: {
brand: ["kia"],
model: ["c'eed"]
}
}
ADF is mostly used to move data from one place to another and to manage ELT process.
So my use case in this scenario would be:
1) copy raw data with ADF to ADLS from sources
2) perform transformations with Azure data lake analytics and save output to the new file
3) import file into power bi (if you do not have analysis service to create tabular model)
Using node-recurly, I can create a subscription object and pass it to recurly.subscriptions.create call:
const subscription = {
plan_code: plan.code,
currency: 'USD',
account: {
account_code: activationCode,
first_name: billingInfo.first_name,
last_name: billingInfo.last_name,
email: billingInfo.email,
billing_info: {
token_id: paymentToken,
},
},
};
I would also like to add subscription_add_ons property, which, looking at the documentation, supposed to be an array of add-ons. I tried passing it like this:
subscription_add_ons: [
{
add_on_code: shippingMethod.servicelevel_token,
unit_amount_in_cents: parseFloat(shippingMethod.amount) * 100,
},
],
The server returned an error:
Tag <subscription_add_ons> must consist only of sub-tags named
<subscription_add_on>
I attempted this:
subscription_add_ons: [
{
subscription_add_on: {
add_on_code: shippingMethod.servicelevel_token,
unit_amount_in_cents: parseFloat(shippingMethod.amount) * 100,
},
},
],
Got back this error:
What's the proper format to pass subscription add on in this scenario?
The proper format is:
subscription_add_ons: {
subscription_add_on: [{
add_on_code: shippingMethod.servicelevel_token,
unit_amount_in_cents: parseFloat(shippingMethod.amount) * 100,
}],
},
I ended up doing this which works whether you have 1 add-on or multiple add-ons. subscription_add_ons is an array which can contain 1 or more subscription add ons. I then send over the details (along with other info) in the subscription update call. This is similar to what you attempted in your original post so I'm not sure why that didn't work for you.
details.subscription_add_ons = [
{ subscription_add_on: {add_on_code: "stream", quantity: 3} },
{ subscription_add_on: {add_on_code: "hold", quantity: 2} }
];
Coverage Model.
var CoverageSchema = new Schema({
module : String,
source: String,
namespaces: [{
name: String,
types: [{
name: String,
functions: [{
name: String,
coveredBlocks: Number,
notCoveredBlocks: Number
}]
}]
}]
});
I need coveredBlocks aggregations on every level:
*Module: {moduleBlocksCovered}, // SUM(blocksCovered) GROUP BY module, source
**Namespaces: [{nsBlocksCovered}] // SUM(blocksCovered) GROUP BY module, source, ns
****Types: [{typeBlocksCovered}] // SUM(blocksCovered) BY module, source, ns, type
How do I get this result with Coverage.aggregate in Mongoose ?
{
module: 'module1',
source: 'source1',
coveredBlocks: 7, // SUM of all functions in module
namespaces:[
name: 'ns1',
nsBlocksCovered: 7, // SUM of all functions in namespace
types:[
{
name: 'type1',
typeBlocksCovered: 7, // SUM(3, 4) of all function in type
functions[
{name: 'func1', blocksCovered: 3},
{name:'func2', blocksCovered: 4}]
}
]
]
}
My ideas is to deconstruct everything using $unwind then reconstruct the document back again using group and projection.
aggregate flow:
//deconstruct functions
unwind(namesapces)
unwind(namespaces.types)
unwind(namespace.types.functions)
//cal typeBlocksCovered
group module&source ,ns,type to sum functions blocksCovered->typeBlocksCovered + push functions back to types
project to transform fields to be easier for next group
// cal nsBlocksCovered
group module&source ,ns to sum typeBlocksCovered -> nsBlocksCovered) + push types back to ns
project to transform fields to be easier for next group
// cal coveredBlocks
group module&source to sum nsBlocksCovered -> coveredBlocks
project to transform fields to match your mongoose docs
My sample query with mongo shell syntax and its seem working , guess is you are using collection name "Coverage"
db.Coverage.aggregate([
{"$unwind":("$namespaces")}
,{"$unwind":("$namespaces.types")}
,{"$unwind":("$namespaces.types.functions")}
,{"$group": {
_id: {module:"$module", source:"$source", nsName: "$namespaces.name", typeName : "$namespaces.types.name"}
, typeBlocksCovered : { $sum : "$namespaces.types.functions.blocksCovered"}
, functions:{ "$push": "$namespaces.types.functions"}}}
,{"$project" :{module:"$_id.module", source:"$_id.source"
,namespaces:{
name:"$_id.nsName"
,types : { name: "$_id.typeName",typeBlocksCovered : "$typeBlocksCovered" ,functions: "$functions"}
}
,_id:0}}
,{"$group": {
_id: {module:"$module", source:"$source", nsName: "$namespaces.name"}
, nsBlocksCovered : { $sum : "$namespaces.types.typeBlocksCovered"}
, types:{ "$push": "$namespaces.types"}}}
,{"$project" :{module:"$_id.module", source:"$_id.source"
,namespaces:{
name:"$_id.nsName"
,nsBlocksCovered:"$nsBlocksCovered"
,types : "$types"
}
,_id:0}}
,{"$group": {
_id: {module:"$module", source:"$source"}
, coveredBlocks : { $sum : "$namespaces.nsBlocksCovered"}
, namespaces:{ "$push": "$namespaces"}}}
,{"$project" :{module:"$_id.module", source:"$_id.source", coveredBlocks : "$coveredBlocks", namespaces: "$namespaces",_id:0}}
])