I am storing in my mongodb collection the video id of a video served in the cloud. When my front end makes an API call for that video, I want to return the formatted URL based on that video id. I have a function that does this, but I can't get it to work with $addFields in and aggregate pipeline:
My documents look like this:
{
"date" : ISODate("2018-03-30T00:00:00.000+0000"),
"title" : "Tips and Tricks",
"video_id" : "13966740",
}
...and I want my API call to return them with an extra field, "image_url":
{
"date" : ISODate("2018-03-30T00:00:00.000+0000"),
"title" : "Tips and Tricks",
"video_id" : "13966740",
"image_url" : "https://myhostingservice.com/13966740/preview_image",
}
This is what I'm trying in models/video.js:
const hostingservice = require('myhostingservicehelperfunctions');
module.exports.getVideo = function (callback) {
videoCollection.aggregate(
[
{ $match: { } },
{ $addFields: {
image_url: hostingservice.createImageURL("$video_id")
}
},
{ $sort: {
'date' : -1 }
}
],
callback);
};
The helper function simply takes the string param and returns a string.
myhostingservicehelperfunctions.js:
module.exports.createImageURL = function ImageURL(video_id){
return 'https://myhostingservice.com/' + video_id + '/preview_image';
};
My front end receives all the correct data, but the value of image_url is "https://myhostingservice.com/$video_id/preview_image", which makes me think my function ran, but it was passed the actual string "$video_id", not the value of the key, video_id. Look closely, the URL contains "$video_id", not "13966740" from the example above.
What am I doing wrong here?
You can't concatenate strings like that in mongoDB aggregation, you'll need to concatenate strings using the concat operator, so the function "createImageURL" should look something like this,
module.exports.createImageURL = function ImageURL(){
return { $concat: [ "https://myhostingservice.com/", "$video_id", "/preview_image" ] };
};
Related
I have a mongoose schema of mixed types like so:
const user = mongoose.Schema({
...
"links" : []
...
After populating this schema, I ended up with data like so:
[
[
{
"step1": "post-url-google", // This field is unique for each entry
"step2": {
"title": "Heading 1",
"likes": 4
}
},
],
[
{
"step1": "post-url-microsoft",
"step2": {
"title": "Heading 1",
"likes": 1
}
},
{
"step1": "post-url-apple",
"step2": {
"title": "Heading 2",
"likes": 6 // I want to update this to 7
}
}
]
]
What I want to achieve is to update the "step1": "post-url-apple" field from 6 to have a likes of 7
So I tried using the User.save() function like so:
let user = await User.findOne({"_id" : "some_id"})
user.links[1].some(object => {
if (object.step1 === "post-url-apple") {
object.step2.likes = 7
(async function (){
user.save() // I also did error handling
})()
return
}
})
This method works fine and the user gets updated but it keeps throwing ParallelSaveError possibly because I am calling the save() function in parallel on the same user instance in some other parts of my code.
So I decided to use the User.findOneAndUpdate() method, but my queries keep failing when using the mongodb dot notation $[<identifier>], obviously because I don't know how to use it properly.
Like so:
let update = {
"$set" : {
"links.$[index1].$[index2].step2.likes" : 7,
}
}
let conditions = {
arrayFilters : [
{"index1" : 1},
{"index2.step1" : "post-url-apple"}
]
}
try {
let result = await Users.findOneAndUpdate({"_id" : "some_id"}, update, conditions)
console.log(result)
} catch (err) {
console.log(err)
}
For all good reasons, I'm not hiting the catch block but the update was equally not successful
How do I achieve updating the "step1": "post-url-apple" likes field to 7 using findOneAndUpdate?
Thank you.
In arrayFilters you should define the conditions to be applied to all the array elements, not the index
If you are sure, you always update the second array element (index = 1) of the outer array, then you can use the dot notation for the outer array, and for the inner array you can use the array filters to get the element that has step1 = 'post-url-apple'
your code may look something like that
let update = {
"$set" : {
'links.1.$[item].step2.likes': 7 // here we used links.1 to access the second element of the outer array
}
}
let conditions = {
arrayFilters : [
{ 'item.step1' : 'post-url-apple' } // item here is the element in the inner array that has step1 = post-url-apple
]
}
then do your update query
hope it helps
I am having a problem searching for a key of a nested object.
I have search criteria object that may or may not have certain fields I'd like to search on.
The way I'm solving this is to use conditional statements to append to a "match criteria" object that gets passed to the aggregate $match operator. it works well until I need to match to something inside a nested object.
Here is a sample document structure
{
name: string,
dates: {
actived: Date,
suspended: Date
},
address : [{
street: string,
city: string,
state: string,
zip: string
}]
};
My criteria object is populated thru a UI and passed a JSON that looks similar to this:
{
"name": "",
"state": ""
}
And although I can explicitly use "dates.suspended" without issue -
when I try to append address.state to my search match criteria - I get an error.
module.exports.search = function( criteria, callback )
let matchCriteria = {
"name": criteria.name,
"dates.suspended": null
};
if ( criteria.state !== '' ) {
// *** PROBLEM HAPPENS HERE *** //
matchCriteria.address.state = criteria.state;
}
User.aggregate([
{ "$match": matchCriteria },
{ "$addFields": {...} },
{ "$project": {...} }
], callback );
}
I get the error:
TypeError: Cannot set property 'state' of undefined
I understand that I'm specifying 'address.state' when 'address' doesn't exist yet - but I am unclear what my syntax would be surely it woulnd't be matchCriteria['address.state'] or "matchCriteria.address.state"
Is there a better way to do conditional filtering?
For search in Nested Object, You have to use unwind
A query that help you :
//For testing declare criteria as const
let criteria = {name : 'name', 'state' : 'state'};
let addressMatch = {};
let matchCriteria = {
"name": criteria.name,
"dates.suspended": null
};
if ( criteria.state) {
addressMatch = { 'address.state' : criteria.state };
}
db.getCollection('user').aggregate([{
$match :matchCriteria,
},{$unwind:'$address'},
{$match : addressMatch}
])
Firstly check for address, and then access the property as shown:
if(matchCriteria['address']) {
matchCriteria['address']['state'] = criteria['state'];
}
else {
//otherwise
}
This should fix it:
matchCriteria['address.state'] = criteria.state;
In the following example, assume the document is in the db.people collection.
How to remove the 3rd element of the interests array by it's index?
{
"_id" : ObjectId("4d1cb5de451600000000497a"),
"name" : "dannie",
"interests" : [
"guitar",
"programming",
"gadgets",
"reading"
]
}
This is my current solution:
var interests = db.people.findOne({"name":"dannie"}).interests;
interests.splice(2,1)
db.people.update({"name":"dannie"}, {"$set" : {"interests" : interests}});
Is there a more direct way?
There is no straight way of pulling/removing by array index. In fact, this is an open issue http://jira.mongodb.org/browse/SERVER-1014 , you may vote for it.
The workaround is using $unset and then $pull:
db.lists.update({}, {$unset : {"interests.3" : 1 }})
db.lists.update({}, {$pull : {"interests" : null}})
Update: as mentioned in some of the comments this approach is not atomic and can cause some race conditions if other clients read and/or write between the two operations. If we need the operation to be atomic, we could:
Read the document from the database
Update the document and remove the item in the array
Replace the document in the database. To ensure the document has not changed since we read it, we can use the update if current pattern described in the mongo docs
You can use $pull modifier of update operation for removing a particular element in an array. In case you provided a query will look like this:
db.people.update({"name":"dannie"}, {'$pull': {"interests": "guitar"}})
Also, you may consider using $pullAll for removing all occurrences. More about this on the official documentation page - http://www.mongodb.org/display/DOCS/Updating#Updating-%24pull
This doesn't use index as a criteria for removing an element, but still might help in cases similar to yours. IMO, using indexes for addressing elements inside an array is not very reliable since mongodb isn't consistent on an elements order as fas as I know.
in Mongodb 4.2 you can do this:
db.example.update({}, [
{$set: {field: {
$concatArrays: [
{$slice: ["$field", P]},
{$slice: ["$field", {$add: [1, P]}, {$size: "$field"}]}
]
}}}
]);
P is the index of element you want to remove from array.
If you want to remove from P till end:
db.example.update({}, [
{ $set: { field: { $slice: ["$field", 1] } } },
]);
Starting in Mongo 4.4, the $function aggregation operator allows applying a custom javascript function to implement behaviour not supported by the MongoDB Query Language.
For instance, in order to update an array by removing an element at a given index:
// { "name": "dannie", "interests": ["guitar", "programming", "gadgets", "reading"] }
db.collection.update(
{ "name": "dannie" },
[{ $set:
{ "interests":
{ $function: {
body: function(interests) { interests.splice(2, 1); return interests; },
args: ["$interests"],
lang: "js"
}}
}
}]
)
// { "name": "dannie", "interests": ["guitar", "programming", "reading"] }
$function takes 3 parameters:
body, which is the function to apply, whose parameter is the array to modify. The function here simply consists in using splice to remove 1 element at index 2.
args, which contains the fields from the record that the body function takes as parameter. In our case "$interests".
lang, which is the language in which the body function is written. Only js is currently available.
Rather than using the unset (as in the accepted answer), I solve this by setting the field to a unique value (i.e. not NULL) and then immediately pulling that value. A little safer from an asynch perspective. Here is the code:
var update = {};
var key = "ToBePulled_"+ new Date().toString();
update['feedback.'+index] = key;
Venues.update(venueId, {$set: update});
return Venues.update(venueId, {$pull: {feedback: key}});
Hopefully mongo will address this, perhaps by extending the $position modifier to support $pull as well as $push.
I would recommend using a GUID (I tend to use ObjectID) field, or an auto-incrementing field for each sub-document in the array.
With this GUID it is easy to issue a $pull and be sure that the correct one will be pulled. Same goes for other array operations.
For people who are searching an answer using mongoose with nodejs. This is how I do it.
exports.deletePregunta = function (req, res) {
let codTest = req.params.tCodigo;
let indexPregunta = req.body.pregunta; // the index that come from frontend
let inPregunta = `tPreguntas.0.pregunta.${indexPregunta}`; // my field in my db
let inOpciones = `tPreguntas.0.opciones.${indexPregunta}`; // my other field in my db
let inTipo = `tPreguntas.0.tipo.${indexPregunta}`; // my other field in my db
Test.findOneAndUpdate({ tCodigo: codTest },
{
'$unset': {
[inPregunta]: 1, // put the field with []
[inOpciones]: 1,
[inTipo]: 1
}
}).then(()=>{
Test.findOneAndUpdate({ tCodigo: codTest }, {
'$pull': {
'tPreguntas.0.pregunta': null,
'tPreguntas.0.opciones': null,
'tPreguntas.0.tipo': null
}
}).then(testModificado => {
if (!testModificado) {
res.status(404).send({ accion: 'deletePregunta', message: 'No se ha podido borrar esa pregunta ' });
} else {
res.status(200).send({ accion: 'deletePregunta', message: 'Pregunta borrada correctamente' });
}
})}).catch(err => { res.status(500).send({ accion: 'deletePregunta', message: 'error en la base de datos ' + err }); });
}
I can rewrite this answer if it dont understand very well, but I think is okay.
Hope this help you, I lost a lot of time facing this issue.
It is little bit late but some may find it useful who are using robo3t-
db.getCollection('people').update(
{"name":"dannie"},
{ $pull:
{
interests: "guitar" // you can change value to
}
},
{ multi: true }
);
If you have values something like -
property: [
{
"key" : "key1",
"value" : "value 1"
},
{
"key" : "key2",
"value" : "value 2"
},
{
"key" : "key3",
"value" : "value 3"
}
]
and you want to delete a record where the key is key3 then you can use something -
db.getCollection('people').update(
{"name":"dannie"},
{ $pull:
{
property: { key: "key3"} // you can change value to
}
},
{ multi: true }
);
The same goes for the nested property.
this can be done using $pop operator,
db.getCollection('collection_name').updateOne( {}, {$pop: {"path_to_array_object":1}})
For example, I have some documents that look like this:
{
id: 1
name: "foo"
}
And I want to append another string to the current name field value.
I tried the following using Mongoose, but it didn't work:
Model.findOneAndUpdate({ id: 1 }, { $set: { name: +"bar" } }, ...);
Edit:
From Compatibility Changes in MongoDB 3.6:
MongoDB 3.6.1 deprecates the snapshot query option.
For MMAPv1, use hint() on the { _id: 1} index instead to prevent a cursor from returning a document more than once if an intervening write operation results in a move of the document.
For other storage engines, use hint() with { $natural : 1 } instead.
Original 2017 answer:
You can't refer to the values of the document you want to update, so you will need one query to retrieve the document and another one to update it. It looks like there's a feature request for that in OPEN state since 2016.
If you have a collection with documents that look like:
{ "_id" : ObjectId("590a4aa8ff1809c94801ecd0"), "name" : "bar" }
Using the MongoDB shell, you can do something like this:
db.test.find({ name: "bar" }).snapshot().forEach((doc) => {
doc.name = "foo-" + doc.name;
db.test.save(doc);
});
The document will be updated as expected:
{ "_id" : ObjectId("590a4aa8ff1809c94801ecd0"), "name": "foo-bar" }
Note the .snapshot() call.
This ensures that the query will not return a document multiple times because an intervening write operation moves it due to the growth in document size.
Applying this to your Mongoose example, as explained in this official example:
Cat.findById(1, (err, cat) => {
if (err) return handleError(err);
cat.name = cat.name + "bar";
cat.save((err, updatedCat) => {
if (err) return handleError(err);
...
});
});
It's worth mentioning that there's a $concat operator in the aggregation framework, but unfortunately you can't use that in an update query.
Anyway, depending on what you need to do, you can use that together with the $out operator to save the results of the aggregation to a new collection.
With that same example, you will do:
db.test.aggregate([{
$match: { name: "bar" }
}, {
$project: { name: { $concat: ["foo", "-", "$name"] }}
}, {
$out: "prefixedTest"
}]);
And a new collection prefixedTest will be created with documents that look like:
{ "_id" : ObjectId("XXX"), "name": "foo-bar" }
Just as a reference, there's another interesting question about this same topic with a few answers worth reading: Update MongoDB field using value of another field
If this is still relevant, I have a solution for MongoDB 4.2.
I had the same problem where "projectDeadline" fields of my "project" documents were Array type (["2020","12","1"])
Using Robo3T, I connected to my MongoDB Atlas DB using SRV link. Then executed the following code and it worked for me.
Initial document:
{
_id : 'kjnolqnw.KANSasdasd',
someKey : 'someValue',
projectDeadline : ['2020','12','1']
}
CLI Command:
db
.getCollection('mainData')
.find({projectDeadline: {$not: {$eq: "noDeadline"}}})
.forEach((doc) => {
var deadline = doc.projectDeadline;
var deadlineDate = new Date(deadline);
db
.mainData
.updateOne({
_id: doc._id},
{"$set":
{"projectDeadline": deadlineDate}
}
)}
);
Resulting document:
{
_id : 'kjnolqnw.KANSasdasd',
someKey : 'someValue',
projectDeadline : '2020-12-01 21:00:00.000Z'
}
I'm trying to figure out how Mongoose and MongoDB works... I'm really new to them, and I can't seem to figure how to return values based on a find statement, where some of the given parameters in the query possible are null - is there an attribute I can set for this or something?
To explain it further, I have a web page that has different input fields that are used to search for a company, however they're not all mandatory.
var Company = mongoose.model('Company');
Company.find({companyName: req.query.companyName, position: req.query.position,
areaOfExpertise: req.query.areaOfExpertise, zip: req.query.zip,
country: req.query.country}, function(err, docs) {
res.json(docs);
});
By filling out all the input fields on the webpage I get a result back, but only that specific one which matches. Let's say I only fill out country, it returns nothing because the rest are empty, but I wish to return all rows which are e.g. in Germany. I hope I expressed myself clearly enough.
You need to wrap the queries with the $or logic operator, for example
var Company = mongoose.model('Company');
Company.find(
{
"$or": [
{ "companyName": req.query.companyName },
{ "position": req.query.position },
{ "areaOfExpertise": req.query.areaOfExpertise },
{ "zip": req.query.zip },
{ "country": req.query.country }
]
}, function(err, docs) {
res.json(docs);
}
);
Another approach would be to construct a query that checks for empty parameters, if they are not null then include it as part of the query. For example, you can just use the req.query object as your query assuming the keys are the same as your document's field, as in the following:
/*
the req.query object will only have two parameters/keys e.g.
req.query = {
position: "Developer",
country: "France"
}
*/
var Company = mongoose.model('Company');
Company.find(req.query, function(err, docs) {
if (err) throw err;
res.json(docs);
});
In the above, the req.query object acts as the query and has an implicit logical AND operation since MongoDB provides an implicit AND operation when specifying a comma separated list of expressions. Using an explicit AND with the $and operator is necessary when the same field or operator has to be specified in multiple expressions.
If you are after a query that satisfies both logical AND and OR i.e. return all documents that match the conditions of both clauses for example given a query for position AND country OR any other fields then you may tweak the query to:
var Company = mongoose.model('Company');
Company.find(
{
"$or": [
{ "companyName": req.query.companyName },
{
"position": req.query.position,
"country": req.query.country
},
{ "areaOfExpertise": req.query.areaOfExpertise },
{ "zip": req.query.zip }
]
}, function(err, docs) {
res.json(docs);
}
);
but then again this could be subject to what query parameters need to be joined as mandatory etc.
I simply ended up deleting the parameters in the query in case they were empty. It seems all the text fields in the submit are submitted as "" (empty). Since there are no such values in the database, it would return nothing. So simple it never crossed my mind...
Example:
if (req.query.companyName == "") {
delete req.query.companyName;
}