After struggling with:
mongoose to determine update-upsert is doing insert or update
I'm still at a loss for determining whether and update or insert occurred with something like this:
let updateResult = await myDB.updateOne(
searchKey,
newData,
{ upsert: true }
);
updateResult contains:
{ n: 1, nModified: 1, ok: 1 }
nModified does not seem to indicate whether the operation resulted in an update or insert but rather it seems to have some other undertermined meaning.
$ mongod --version
db version v4.4.18
Build Info: {
"version": "4.4.18",
"gitVersion": "8ed32b5c2c68ebe7f8ae2ebe8d23f36037a17dea",
"openSSLVersion": "OpenSSL 1.1.1f 31 Mar 2020",
"modules": [],
"allocator": "tcmalloc",
"environment": {
"distmod": "ubuntu2004",
"distarch": "x86_64",
"target_arch": "x86_64"
}
}
"mongoose": "^5.10.13",
if you are using mongoose then you will get below result when update
{
"acknowledged": true,
"modifiedCount": 1, // if document is updated
"upsertedId": null, // id of new inserted document (if upsert)
"upsertedCount": 0, // count of inserted documents
"matchedCount": 1 // count of documents that match with condition for update
}
Related
I have aerospike 6.0.0 installed on server and being used as cache. Also I am using Node.js client 5.2.0 to connect to the database.The aerospike nodes are configured to use TLS. The config of the namespace I am using is:
namespace application_cache {
memory-size 5G
replication-factor 2
default-ttl 1h
nsup-period 1h
high-water-memory-pct 80
stop-writes-pct 90
storage-engine memory
}
The config I am passing while connecting to the Db is:
"config": {
"captureStackTraces":true,
"connTimeoutMs": 10000,
"maxConnsPerNode": 1000,
"maxCommandsInQueue" : 300,
"maxCommandsInProcess" : 100,
"user": "test",
"password": "test",
"policies": {
"read": {
"maxRetries": 3,
"totalTimeout": 0,
"socketTimeout": 0
},
"write": {
"maxRetries": 3,
"totalTimeout": 0,
"socketTimeout": 0
},
"info": {
"timeout": 10000
}
},
"tls": {
"enable": true,
"cafile": "./certs/ca.crt",
"keyfile": "./certs/server.key",
"certfile": "./certs/server.crt"
},
"hosts": [
{
"addr": "-----.com",
"port": 4333,
"tlsname": "----"
},
{
"addr": "---.com",
"port": 4333,
"tlsname": "-----"
},
{
"addr": "----.com",
"port": 4333,
"tlsname": "-----"
}
]
}
This is the function I am using to write to a set in this namespace:
async function updateApp(appKey, bins) {
let result = {};
try {
const writePolicy = new Aerospike.WritePolicy({
maxRetries: 3,
socketTimeout: 0,
totalTimeout: 0,
exists: Aerospike.policy.exists.CREATE_OR_UPDATE,
key: Aerospike.policy.key.SEND
});
const key = new Aerospike.Key(namespace, set, appKey);
let meta = {
ttl: 3600
};
let bins = {};
result = await asc.put(key, bins, meta, writePolicy);
} catch (err) {
console.error("Aerospike Error:", JSON.stringify({params: {namespace, set, appKey}, err: {code: err.code, message: err.message}}));
return err;
}
return result;
}
It works most of the time but once in a while I get this error:
Aerospike Error: {"params":{"namespace":"application_cache","set":"apps","packageName":"com.test.application"},"err":{"code":-6,"message":"TLS write failed: -2 34D53CA5C4E0734F 10.57.49.180:4333"}}
Every record in this set has about 12-15 bins and the size of the record varies between 300Kb to 1.5Mb. The bigger the size of the record the higher chance of this error showing up. Has anyone faced this issue before and what could be causing this?
I answered this on the community forum but figured I'd add it here as well.
Looking at the server error codes a code: 6 is AS_ERR_BIN_EXISTS. Your write policy is setup with exists: Aerospike.policy.exists.CREATE_OR_UPDATE but the options for exists are:
Name
Description
IGNORE
Write the record, regardless of existence. (I.e. create or update.)
CREATE
Create a record, ONLY if it doesn't exist.
UPDATE
Update a record, ONLY if it exists.
REPLACE
Completely replace a record, ONLY if it exists.
CREATE_OR_REPLACE
Completely replace a record if it exists, otherwise create it.
My guess is that it's somehow defaulting to a CREATE only causing it to throw that error. Try updating your write policy to use exists: Aerospike.policy.exists.IGNORE and see if you still get that error.
Below is an example of a document in a User collection below.
{
"_id" : 1,
"username" : bob,
"pause" : true,
"pause_date" : ISODate("2021-07-16T07:13:48.680Z"),
"learnt_item" : [
{
"memorized" : false,
"character" : "一",
"next_review" : ISODate("2021-07-20T11:02:44.979Z")
},
{
"memorized" : false,
"character" : "二",
"next_review" : ISODate("2021-07-20T11:02:44.979Z")
},
...
]
}
I need to update all the nested document in "learnt_item" if the "memorized" field is false.
The updates are:
"pause_date" to Null
"pause" to False
Update the ISOdate in "next_review" based on the duration that has passed between "pause_date" and the current time.
E.g. pause_date is 4 hours ago, then I want to add 4 hours to the "next_review"
I was able to achieve 1 & 2 using findOneAndUpdate with arrayFilters and also tested no.3 by updating the "next_review" field with a current date to make sure it is updating correctly.
User.findOneAndUpdate({"_id": req.user._id},
{$set:{"learnt_item.$[elem].next_review": DateTime.local(),"pause_date": null, "pause": value }},
{new:true, arrayFilters: [{"elem.memorized": false}]}).exec((err, doc) =>{if (err){res.send(err)} else {res.send(doc)}});
I was thinking of using the $add aggregation operator to increase the date base
"learnt_item.$[elem].next_review": {$add: ["$learnt_item.$[elem].next_review","$pause_date"]}
However, according to the documentation, arrayFilters is not available for updates that use an aggregation pipeline.
Is there another way more efficient way that I can update the ISOdate?
If you are running MongoDB 4.2 or later you can use a pipeline as the second parameter for the update function, this way you can use the operator $map with $cond to find the entries where the property memorized is equal to false and then add 4 days in milliseconds to the next_review date:
db.collection.update({
"_id": 1
},
[
{
$set: {
"pause_date": null,
"pause": false,
"learnt_item": {
$map: {
input: "$learnt_item",
as: "item",
in: {
$cond: [
{
$eq: [
"$$item.memorized",
false
]
},
{
memorized: "$$item.memorized",
character: "$$item.character",
next_review: {
$add: [
"$$item.next_review",
345600000
]
}
},
"$$item"
]
}
}
},
}
}
],
{
new: true,
});
You can check a running example here: https://mongoplayground.net/p/oHh1JWiP8vs
So I'm attempting to find all records who have a field set and isn't null.
I try using $exists, however according to the MongoDB documentation, this query will return fields who equal null.
$exists does match documents that contain the field that stores the null value.
So I'm now assuming I'll have to do something like this:
db.collection.find({ "fieldToCheck" : { $exists : true, $not : null } })
Whenever I try this however, I get the error [invalid use of $not] Anyone have an idea of how to query for this?
Use $ne (for "not equal")
db.collection.find({ "fieldToCheck": { $ne: null } })
Suppose we have a collection like below:
{
"_id":"1234"
"open":"Yes"
"things":{
"paper":1234
"bottle":"Available"
"bottle_count":40
}
}
We want to know if the bottle field is present or not?
Ans:
db.products.find({"things.bottle":{"$exists":true}})
i find that this works for me
db.getCollection('collectionName').findOne({"fieldName" : {$ne: null}})
This comment is written in 2021 and applies for MongoDB 5.X and earlier versions.
If you value query performance never use $exists (or use it only when you have a sparse index over the field that is queried. the sparse index should match the criteria of the query, meaning, if searching for $exists:true, the sparse index should be over field:{$exist:true} , if you are querying where $exists:true the sparse index should be over field:{$exist:false}
Instead use :
db.collection.find({ "fieldToCheck": { $ne: null } })
or
db.collection.find({ "fieldToCheck": { $eq: null } })
this will require that you include the fieldToCheck in every document of the collection, however - the performance will be vastly improved.
db.<COLLECTION NAME>.find({ "<FIELD NAME>": { $exists: true, $ne: null } })
In my case, i added new field isDeleted : true to only fields that are deleted.
So for all other records there was no isDeleted field, so i wanted to get all the fields that isDeleted either does not exist or false. So query is
.find({ isDeleted: { $ne: true } });
I Tried to convert it into boolean condition , where if document with
table name already exist , then it will append in the same document ,
otherwise it will create one .
table_name is the variable using which i am trying to find the document
query = { table_name : {"$exists": "True"}}
result = collection.find(query)
flag = 0
for doc in result:
collection.update_one({}, { "$push" : { table_name : {'name':'hello'} } } )
flag = 1
if (flag == 0):
collection.insert_one({ table_name : {'roll no' : '20'}})
aggregate example
https://mongoplayground.net/p/edbKil4Zvwc
db.collection.aggregate([
{
"$match": {
"finishedAt": {
"$exists": true
}
}
},
{
"$unwind": "$tags"
},
{
"$match": {
"$or": [
{
"tags.name": "Singapore"
},
{
"tags.name": "ABC"
}
]
}
},
{
"$group": {
"_id": null,
"count": {
"$sum": 1
}
}
}
])
I udpated my mongo instance from version 2.4 to 3.4 and all of my update queries stopped working where I was passing $ as index.
If I pass static 0 or 1 in the query it works fine, but earlier syntax of $ won't work at all.
Below is my query :
db.collection('users').update({"email": "u1#u1.com","companies":{"$elemMatch":{"id":"1487006991927"}}},
{
$set: {
"companies.$.details" : {"company_name":"hey updated"}
}
});
Response that I get :
{ result: { _t: 'UpdateResponse', ok: 1, n: 1, nModified: 1 },
This worked perfectly while I was on mongo vesrion 2.4 but not anymore. I can't always pass static 0 / 1 or index, what is the right way to do it ?
Also to note : Response says that 1 record was modified, but nothing was modified actually.
{
"_id": "589aa3509a248a3d7a01b784",
"businessAndPersonal": "true",
"companies": [
{
"details": {
"company_name": "afsfhey updated"
},
"locations": [],
"websites": [],
"id": "1487006991927"
},
{
"details": {
"company_name": "hey updated"
},
"locations": [],
"websites": [],
"id": "1487007435955"
}
]
}
Thanks in advance
Answer for my own question
I am using CosmosDB mongodb service as far they doesn't support positional operator of mongodb
Here is link that has discussion on positional array update via '$' query support
i have a Mongodb collection named "EVENTS" and in the collection i have an object of array which looks like this:
{
"Events":[
{
"_id":"53ae59883d0e63aa77f7b5b2",
"Title":"Title Blank",
"Desc":"Description Blank",
"Date":"2014-06-04 00:30",
"Link":"http://googleparty.com",
"Event":"Victoria Centre",
"dateCreated":"28/6/2014 06:58"
},
{
"_id":"53ae59883d0e63aa77f7b5b3",
"Title":"Hello World",
"Desc":"hello",
"Date":"2014-06-04 00:30",
"Link":"http://linkedinparty.com",
"Event":"social",
"dateCreated":"30/2/2014 11:10"
}
]
}
how would i delete an object by id in node.js so " delete(53ae59883d0e63aa77f7b5b2)" will yield this:
{
"Events":[
{
"_id":"53ae59883d0e63aa77f7b5b3",
"Title":"Hello World",
"Desc":"hello",
"Date":"2014-06-04 00:30",
"Link":"http://linkedinparty.com",
"Event":"social",
"dateCreated":"30/2/2014 11:10"
}
]
}
Regards
If all you really want to do is "empty" the array then you just need to use the $set operator with an .update() and "set" the array as an empty one:
db.collection.update({},{ "$set": { "Events": [] } },{ "mutli": true})
So the .update() operation takes a "query" to select the documents in your collection, a blank query as shown selects everything. The "update" section contains the $set operation that just replaces the current "Events" field with an empty array.
The "multi" option there makes sure this is applied to every document that matches. The default is false and will only update the first document that matches.
For more specific operations removing selected array elements, look at the $pull operator. Your edit shows now that this is what you want to do:
db.collection.update(
{ "Events._id": ObjectId("53ae59883d0e63aa77f7b5b2") },
{ "$pull": { "Events": { "_id": ObjectId("53ae59883d0e63aa77f7b5b2") } } }
)
But your inclusion of arrays with _id fields seems to indicate that you are using mongoose, so the ObjectId values are cast automatically:
Model.update(
{ "Events._id": "53ae59883d0e63aa77f7b5b2" },
{ "$pull": { "Events": { "_id": "53ae59883d0e63aa77f7b5b2" } } },
function(err,numAffected) {
}
);