On a daily basis, I'm pushing data (time_series) to Elasticsearch. I created an index pattern, and my index have the name: myindex_* , where * is today date (an index pattern has been setup). Thus after a week, I have: myindex_2022-06-20, myindex_2022-06-21... myindex_2022-06-27.
Let's assume my index is indexing products' prices. Thus inside each myindex_*, I have got:
myindex_2022-06-26 is including many products prices like this:
{
"reference_code": "123456789",
"price": 10.00
},
...
myindex_2022-06-27:
{
"reference_code": "123456789",
"price": 12.00
},
I'm using this query to get the reference code and the corresponding prices. And it works great.
const data = await elasticClient.search({
index: myindex_2022-06-27,
body: {
query: {
match: {
"reference_code": "123456789"
}
}
}
});
But, I would like to have a query that if in the index of the date 2022-06-27, there is no data, then it checks, in the previous index 2022-06-26, and so on (until e.g. 10x).
Not sure, but it seems it's doing this when I replace myindex_2022-06-27 by myindex_* (not sure it's the default behaviour).
The issue is that when I'm using this way, I got prices from other index but it seems to use the oldest one. I would like to get the newest one instead, thus the opposite way.
How should I proceed?
If you query with index wildcard, it should return a list of documents, where every document will include some meta fields as _index and _id.
You can sort by _index, to make elastic search return the latest document at position [0] in your list.
const data = await elasticClient.search({
index: myindex_2022-*,
body: {
query: {
match: {
"reference_code": "123456789"
}
}
sort : { "_index" : "desc" },
}
});
I'm trying to create multiple documents based on a filter: if document not found => create it.
After searching a bit I found that the correct way to do so is by using updateMany and setting upsert: true (docs).
This made somewhat sense from the documentation's example but as I understand it the filter modifier would be used for the newly-created document. As in the example:
try {
db.inspectors.updateMany(
{ "Sector" : { $gt : 4 }, "inspector" : "R. Coltrane" },
{ $set: { "Patrolling" : false } },
{ upsert: true }
);
} catch (e) {
print(e);
}
"Inspector" : "R. Coltrane" would be inserted to the newly-created document.
But what if my setOnInsert modifier contains the same field as the one in the filter?
What my code looks like:
//first find the already-created tags
await tagModel.find({"tagName": tags}).select('tagName -_id').exec()
.then(async (result: Tag[])=>{
//create the new tags
const newTags = tags.map((tag: any)=>new tagModel({tagName: tag}));
//now insert only the new tags, filtering out the already-created tags ("result")
await tagModel.updateMany(
{"tagName": result},
{$setOnInsert: newTags} ,
{upsert: true},
(err:any, res:any)=>{
...
At first, result is an empty Array ([]). What is created in my MongoDB database is a new Tag document, but its tagName is the result object. Meaning, it looks like:
{
"_id": {
"$oid": "61659c92c6267fe11963b236"
},
"tagName": {
"$in": []
}
}
So essentially my question is, what am I suppose to do in this case where my update modifier should replace my filter query? Perhaps it's just something bad in my code that makes the updateMany function to malfunction? Or should I replace it with a different function?
This is a Controller in which I'm trying to catch multiple candidates id(ObjectId) and try to store it in the database in the array Candidates. But data is not getting pushed in Candidates column of Array type.
routes.post('/Job/:id',checkAuthenticated,function(req,res){
var candidates=req.body.candidate;
console.log(candidates);
Job.update({_id:req.params.id},{$push:{Appliedby : req.user.username}},{$push:{Candidates:{$each:
candidates}}}
});
Console screens output
[ '5eb257119f2b2f0b4883558b', '5eb2ae1cff3ae7106019ad7e' ] //candidates
you have to do all the update operations ($set, $push, $pull, ...) in one object, and this object should be the second argument passed to the update method after the filter object
{$push:{Appliedby : req.user.username}},{$push:{Candidates:{$each: candidates}}
this will update the Appliedby array only, as the third object in update is reserved for the options (like upsert, new, ....)
you have to do something like that
{ $push: { Appliedby: req.user.username, Candidates: { $each: candidates } } }
then the whole query should be something like that
routes.post('/Job/:id', checkAuthenticated, function (req, res) {
var candidates = req.body.candidate;
console.log(candidates);
Job.update(
{ _id: req.params.id }, // filter part
{ $push: { Appliedby: req.user.username, Candidates: { $each: candidates } } } // update part in one object
)
});
this could do the trick I guess, hope it helps
In the following example, assume the document is in the db.people collection.
How to remove the 3rd element of the interests array by it's index?
{
"_id" : ObjectId("4d1cb5de451600000000497a"),
"name" : "dannie",
"interests" : [
"guitar",
"programming",
"gadgets",
"reading"
]
}
This is my current solution:
var interests = db.people.findOne({"name":"dannie"}).interests;
interests.splice(2,1)
db.people.update({"name":"dannie"}, {"$set" : {"interests" : interests}});
Is there a more direct way?
There is no straight way of pulling/removing by array index. In fact, this is an open issue http://jira.mongodb.org/browse/SERVER-1014 , you may vote for it.
The workaround is using $unset and then $pull:
db.lists.update({}, {$unset : {"interests.3" : 1 }})
db.lists.update({}, {$pull : {"interests" : null}})
Update: as mentioned in some of the comments this approach is not atomic and can cause some race conditions if other clients read and/or write between the two operations. If we need the operation to be atomic, we could:
Read the document from the database
Update the document and remove the item in the array
Replace the document in the database. To ensure the document has not changed since we read it, we can use the update if current pattern described in the mongo docs
You can use $pull modifier of update operation for removing a particular element in an array. In case you provided a query will look like this:
db.people.update({"name":"dannie"}, {'$pull': {"interests": "guitar"}})
Also, you may consider using $pullAll for removing all occurrences. More about this on the official documentation page - http://www.mongodb.org/display/DOCS/Updating#Updating-%24pull
This doesn't use index as a criteria for removing an element, but still might help in cases similar to yours. IMO, using indexes for addressing elements inside an array is not very reliable since mongodb isn't consistent on an elements order as fas as I know.
in Mongodb 4.2 you can do this:
db.example.update({}, [
{$set: {field: {
$concatArrays: [
{$slice: ["$field", P]},
{$slice: ["$field", {$add: [1, P]}, {$size: "$field"}]}
]
}}}
]);
P is the index of element you want to remove from array.
If you want to remove from P till end:
db.example.update({}, [
{ $set: { field: { $slice: ["$field", 1] } } },
]);
Starting in Mongo 4.4, the $function aggregation operator allows applying a custom javascript function to implement behaviour not supported by the MongoDB Query Language.
For instance, in order to update an array by removing an element at a given index:
// { "name": "dannie", "interests": ["guitar", "programming", "gadgets", "reading"] }
db.collection.update(
{ "name": "dannie" },
[{ $set:
{ "interests":
{ $function: {
body: function(interests) { interests.splice(2, 1); return interests; },
args: ["$interests"],
lang: "js"
}}
}
}]
)
// { "name": "dannie", "interests": ["guitar", "programming", "reading"] }
$function takes 3 parameters:
body, which is the function to apply, whose parameter is the array to modify. The function here simply consists in using splice to remove 1 element at index 2.
args, which contains the fields from the record that the body function takes as parameter. In our case "$interests".
lang, which is the language in which the body function is written. Only js is currently available.
Rather than using the unset (as in the accepted answer), I solve this by setting the field to a unique value (i.e. not NULL) and then immediately pulling that value. A little safer from an asynch perspective. Here is the code:
var update = {};
var key = "ToBePulled_"+ new Date().toString();
update['feedback.'+index] = key;
Venues.update(venueId, {$set: update});
return Venues.update(venueId, {$pull: {feedback: key}});
Hopefully mongo will address this, perhaps by extending the $position modifier to support $pull as well as $push.
I would recommend using a GUID (I tend to use ObjectID) field, or an auto-incrementing field for each sub-document in the array.
With this GUID it is easy to issue a $pull and be sure that the correct one will be pulled. Same goes for other array operations.
For people who are searching an answer using mongoose with nodejs. This is how I do it.
exports.deletePregunta = function (req, res) {
let codTest = req.params.tCodigo;
let indexPregunta = req.body.pregunta; // the index that come from frontend
let inPregunta = `tPreguntas.0.pregunta.${indexPregunta}`; // my field in my db
let inOpciones = `tPreguntas.0.opciones.${indexPregunta}`; // my other field in my db
let inTipo = `tPreguntas.0.tipo.${indexPregunta}`; // my other field in my db
Test.findOneAndUpdate({ tCodigo: codTest },
{
'$unset': {
[inPregunta]: 1, // put the field with []
[inOpciones]: 1,
[inTipo]: 1
}
}).then(()=>{
Test.findOneAndUpdate({ tCodigo: codTest }, {
'$pull': {
'tPreguntas.0.pregunta': null,
'tPreguntas.0.opciones': null,
'tPreguntas.0.tipo': null
}
}).then(testModificado => {
if (!testModificado) {
res.status(404).send({ accion: 'deletePregunta', message: 'No se ha podido borrar esa pregunta ' });
} else {
res.status(200).send({ accion: 'deletePregunta', message: 'Pregunta borrada correctamente' });
}
})}).catch(err => { res.status(500).send({ accion: 'deletePregunta', message: 'error en la base de datos ' + err }); });
}
I can rewrite this answer if it dont understand very well, but I think is okay.
Hope this help you, I lost a lot of time facing this issue.
It is little bit late but some may find it useful who are using robo3t-
db.getCollection('people').update(
{"name":"dannie"},
{ $pull:
{
interests: "guitar" // you can change value to
}
},
{ multi: true }
);
If you have values something like -
property: [
{
"key" : "key1",
"value" : "value 1"
},
{
"key" : "key2",
"value" : "value 2"
},
{
"key" : "key3",
"value" : "value 3"
}
]
and you want to delete a record where the key is key3 then you can use something -
db.getCollection('people').update(
{"name":"dannie"},
{ $pull:
{
property: { key: "key3"} // you can change value to
}
},
{ multi: true }
);
The same goes for the nested property.
this can be done using $pop operator,
db.getCollection('collection_name').updateOne( {}, {$pop: {"path_to_array_object":1}})
Is it possible to $addToSet and determine which items were added to the set?
i.e. $addToSet tags to a post and return which ones were actually added
Not really, and not with a single statement. The closest you can get is with the findAndModify() method, and compare the orginal document form to the fields that you submitted in your $addToSet statement:
So considering an initial document:
{
"fields": [ "B", "C" ]
}
And then processing this code:
var setInfo = [ "A", "B" ];
var matched = [];
var doc = db.collection.findAndModify(
{ "_id": "myid" },
{
"$addToSet": { "fields": { "$each": setInfo } }
}
);
doc.fields.forEach(function(field) {
if ( setInfo.indexOf(field) != -1 ) {
matched.push(field);
}
});
return matched;
So that is a basic JavaScript abstraction of the methods and not actually nodejs general syntax for either the native node driver or the Mongoose syntax, but it does describe the basic premise.
So as long as you are using a "default" implementation method that returns the "original" state of the document before it was modified the you can play "spot the difference" as it were, and as is shown in the code example.
But doing this over general "update" operations is just not possible, as they are designed to possibly affect one or more objects and never return this detail.