I have been trying to execute a Bulkwrite query in Mongoose, and I don't know for what reasons It doesn't throw error
let input = [
{
"variantId": "5e1760fbdfaf28038242d676",
"quantity": 5
},
{
"variantId": "5e17e67b73a34d53160c7252",
"quantity": 13
}
]
try{
let bulkArr = [];
for (const i of input) {
bulkArr.push({
updateOne: {
"filter": { "_id": Mongoose.Types.ObjectId(i.variantId) },
"update": { $inc: { "stocks": - i.quantity } }
}
})
}
await Variant.bulkWrite(bulkArr)
}
catch(e){
console.log(e);
}
I have tried to update values not present in the database,
as well as I have tried to update stocks to negative values which is explicitly set to min:0
both of the following query should have thrown an error. But no error was thrown.
Updating non-existing document is not considered as write error. You can console.log the result to see that it simply informs you about the number of matched elements:
let result = await Test.bulkWrite(bulkArr);
console.log(result);
which prints:
...
nMatched: 0,
...
Running $inc with a negative value is still valid - you can do that and even if you have a validation rule like min:0 it will not be applied since $inc gets executed on database server and mongoose doesn't know if your field's value will get negative after this operation or not. The min rule will be applied when you want to insert a new document or replace existing one using .save() method.
How to get an error then ? Try to violate unique key by inserting the same value into unique index - that situation throws an exception explicitly.
Related
I have the following object in my mongodb:
{
"item": {
"id": "/Ed/6wigZ9LTLs2mPDWDzOFD/he0vbUEvQBl2Bga/T8=",
"status": {
"likes": [
"HU0HKFoL2YQuQ2WrYhj0rYoFbRkwJ0EJEf4ML7vAp2Q="
]
}
}
}
Now I want to update my collection, and pull the value "/Ed/6wigZ9LTLs2mPDWDzOFD/he0vbUEvQBl2Bga/T8=" from all documents in the collection who have this value in their item.status.liles array.
When I use the tool Robo 3T, I can put into the command line:
db.getCollection('mycollection').update({"item.status.likes": "/Ed/6wigZ9LTLs2mPDWDzOFD/he0vbUEvQBl2Bga/T8=" }, { "$pull":{"item.status.likes": "/Ed/6wigZ9LTLs2mPDWDzOFD/he0vbUEvQBl2Bga/T8="}})
And it says Updated 0 record(s) in 2ms when I execute the command (which is correct, since there is no document matching).
Now when I do the same in Node.JS code, like this:
let filter = {item.status.likes: '/Ed/6wigZ9LTLs2mPDWDzOFD/he0vbUEvQBl2Bga/T8='};
let obj = {"$pull":{"item.status.likes":"/Ed/6wigZ9LTLs2mPDWDzOFD/he0vbUEvQBl2Bga/T8="}}
collection.updateMany(filter, obj, { upsert: true }, (err, info) => {...});
I get MongoError: Cannot apply $pull to a non-array value instead.
If I change it to:
collection.updateMany({}, obj, { upsert: true }, (err, info) => {...});
I get no error. This leads me to believe that there is a difference between "no filter" and "empty result set", and $pull does not work on the latter.
Why is this? I also tried with different "$pull"-syntax (e.g. {"item":{"status":{"likes"... vs item.status.likes) but this didnt change anything.
The error happens when there is no single document matching the filter because of upsert: true.
It tells mongo to insert a new document with fields from the filter, which makes following document:
{
"item": {
"status": {
"likes": "/Ed/6wigZ9LTLs2mPDWDzOFD/he0vbUEvQBl2Bga/T8="
}
}
}
Then it tries to apply the update to this document calling $pull on the string which results with the error.
The Robo3T version works because the update is upsert: false by default.
I have a Mongoose abTest document that has two fields:
status. This is a string enum and can be of type active, inactive or draft.
validCountryCodes. This is an array of strings enums (GB, EU, AU etc). By default, it will be empty.
In the DB, at any one time, I only want there to be one active abTest for each validCountryCode so I'm performing some validation prior to creating or editing a new abTest.
To do this, I've written a function that attempts to count the number of documents that have a status of active and that contain one of the countryCodes.
The function will then return if the count is more than one. If so, I will throw a validation error.
if (params.status === 'active') {
const activeTestForCountryExists = await checkIfActiveAbTestForCountry(
validCountryCodes,
);
if (params.activeTestForCountryExists) {
throw new ValidationError({
message: 'There can only be one active test for each country code.',
});
}
}
const abTest = await AbTest.create(params);
checkIfActiveAbTestForCountry() looks like this:
const checkIfActiveAbTestForCountry = async countryCodes => {
const query = {
status: 'active',
};
if (
!countryCodes ||
(Array.isArray(countryCodes) && countryCodes.length === 0)
) {
query.validCountryCodes = {
$eq: [],
};
} else {
query.validCountryCodes = { $in: [countryCodes] };
}
const count = await AbTest.countDocuments(query);
return count > 0;
};
The count query should count not only exact array matches, but for any partial matches.
If in the DB there is an active abTest with a validCountryCodes array of ['GB', 'AU',], the attempting to create a new abTest with ['GB' should fail. As there is already a test with GB as a validCountryCode.
Similarly, if there is a test with a validCountryCodes array of ['AU'], then creating a test with validCountryCodes of ['AU,'NZ'] should also fail.
Neither is enforced right now.
How can I do this? Is this possible write a query that checks for this?
I considered iterating over params.validCountryCodes and counting the docs that include each, but this seems like bad practice.
take a look at this MongoDB documantation.
As I understood what you need is to find out if there is any document that contains at least one of the specified countryCodes and it has active status. then your query should look like this:
{
status: 'active',
$or: [
{ validCountryCodes: countryCodes[0] },
{ validCountryCodes: countryCodes[1] },
// ...
]
}
note that counting documents is not an efficient manner to check if a document exists or not, instead use findOne with only one field being projected.
You are using the correct mongo-query for your requirement. Can you verify the actual queries executed from your application is the same? Check here
{ status: 'active', validCountryCodes: { $in: [ countryCodes ] } }
For eg; below query :
{ status: 'active', validCountryCodes: { $in: ['GB' ] } }
should match document :
{ status: 'active', validCountryCodes: ['AU','GB'] }
I am trying to build a "number of visitors" collection in mongoDb using Node.JS backend of my website. The frontend sends the following info to Node.JS backend as JSON.
isUniqueVisitor - 1 if yes, 0 if no
country - standard country code - "JP", "IN", "UK", etc
My database looks like following
{
"today": 2019-06-07,
"uniqueVisitors": {
"count": 230,
"countries": {
"JP": 102,
"IN": 88,
"UK": 30
}
}
}
It works well if I use $inc with fixed values
Eg. $inc: {count: 1} // for string/integers keys
Eg. $inc: {"uniqueVisitors.count": 1} // inside quotes to access key of a JSON
Main issue:
I am not able to access a document name using variable.
Eg. $inc: {`uniqueVisitors.countries[${req.body.country}]`}
This creates an error as backticks can't be used for Mongo.
I tried with
Eg. $inc: {uniqueVisitors["countries"][req.body.country]}
But even this creates error.
I followed the web and found that mongo $set using variables can be realized by passing the required JSON directly to $set. Hence I resorted to code it the following way.
mongoClient.connect(mongoURL, async function (err, db) {
if (err) throw err;
console.log("Database connected");
// Identifying my document with today's date
var myQuery = {
date: getTodayDate()
};
// Defining the JSON to be passed to uniqueVisitors $inc
var uniqueVisitorsInc = {
"uniqueVisitors": {
"count": 0,
"countries": {}
}
};
// Populating the JSON to be passed to uniqueVisitors $inc => essentially asking to increase count by 1 and increase that country's count by 1
uniqueVisitorsInc["uniqueVisitors"]["count"] = 1;
uniqueVisitorsInc["uniqueVisitors"]["countries"][myData.country] = 1;
var newValues = {
$inc: uniqueVisitorsInc
};
await db.collection("visitorStats").update(myQuery, newValues, {upsert: true});
db.close();
});
The above method worked well on editor but threw the following runtime error:
$inc requires numerical values
Basically asking me to pass values to $inc in {var1: 1, var2: 5} pattern.
Please help me bypass this weird situation.
I know I can do a two step process where I read the values first, increment in variable and $set it in Mongo.
But does anyone know how to overcome this situation using $inc?
If this update were hardcoded to update "JP" only, it'd need to look like:
$inc: { "uniqueVisitors.country.JP": 1 }
So you were almost there with the backtick method but change the syntax a bit and keep the : 1 part like so:
$inc: { [`uniqueVisitors.country.${req.body.country}`]: 1 }
Hi I have a fairly large dataset 55K records. I am calculating the moving average for those. Which is the most efficient way to store these results again?
Currently I am doing this. Which leads to extreme amount of records being written one by one. Is there a way for me to write the whole list back in one call if I add the calculated value to the records array?
updated the code with the bulk update. It however fails to update the "MA16" record. Even though I know for sure that there is valid data in the "list" array.
It seems to match the documents but won't update. It will yield a MA16 field in the database that is always null.
Logged output.
deletedCount:0
insertedCount:0
insertedIds:Object {}
matchedCount:150
modifiedCount:0
n:0
nInserted:0
nMatched:150
nModified:0
nRemoved:0
nUpserted:0
ok:1
var bulkUpdateArray = _.map(records, (record, index) => {
return {
updateOne :{
"filter":{_id : record._id},
"update":{$set: {"MA16": list[index]}},
"upsert":true
}
}
});
mongoose.connection.db.collection(req.body.day).bulkWrite(bulkUpdateArray).then(result => {
console.log("Insert result", result);
}).catch(err=>{
//catch the error here
})
You can use BulkWrite to achieve what you want.
Try this:
var bulkUpdateArray = _.map(records, (record, index) => {
return {
updateOne :{
"filter":{_id : record._id},
"update":{$set: {"MA16": list[index]}},
"upsert":true
}
}
})
mongoose.connection.db.collection(req.body.day).bulkWrite(bulkUpdateArray).then(result => {
//check the result of bulk update here
}).catch(err=>{
//catch the error here
})
You can use updateOne operator of bulkWrite.
From official MongoDB docs, bulkWrite has following options:
{ updateOne :
{
"filter" : <document>,
"update" : <document>,
"upsert" : <boolean>,
"collation": <document>,
"arrayFilters": [ <filterdocument1>, ... ]
}
}
Please read MongoDB bulkWrite documentation for more info.
I hope this helps you out.
I have a MongoDB collection with documents that look as follows:
{
"id": 51584,
"tracks": [],
"_id": {
"$oid": "ab5a7... some id ...cc81da0"
}
}
I want to push a single track into the array, so I try the following NodeJS code:
function addTrack(post,callback){
var partyId = post['partyId'], trackId = post['trackId'];
// I checked here that partyId and trackId are valid vars.
db.db_name.update({id: partyId}, { $push: { tracks: [trackId] } }, function(err, added) {
if( err || !added ) {
console.log("Track not added.");
callback(null,added);
}
else {
console.log("Track added to party with id: "+partyId);
callback(null,added);
}
});
}
This returns successfully with the callback that the track was added. However, when I inspect the database manually it is not updated and the array tracks is still empty.
I've tried a lot of different things for the tracks element to be pushed (ie. turning it into an array etc.) but no luck so far.
PS: Perhaps I should note that I'm using MongoLab to host the database.
Any help would be most welcome.
I found my problem, in the addTrack update({id: partyId},.. method partyId was not a string so it didn't find any docs to push to. Thanks to SudoGetBeer for leading me to the solution.
If your posted document is correct(get via find() i.e.):
tracks is a subdocument or embedded document ( http://docs.mongodb.org/manual/tutorial/query-documents/#embedded-documents )
The difference is simple: {} = Document, [] = Array
So if you want to use $push you need to update the tracks field to be an array
Here's how I'm doing it:
// This code occurs inside an async function called editArticle()
const addedTags = ['one', 'two', 'etc']
// ADD NEW TAGS IN MONGO DB
try {
const updateTags = addedTags.reduce((all, tag) => {
all.push(articles.updateOne({ slug: article.slug }, { $push: { tags: tag } }))
return all
}, [])
await Promise.all(updateTags)
} catch (e) {
log('error', 'addTags', e)
throw 'addTags'
}
addTags is the Array of tags, and we need to push them into Mongo DB one at a time so that the document we are pushing into looks like this after:
{
tags: ["existingTag1", "existingTag2", "one", "two", "etc"]
}
If you push an array like in the original question above, it would look like this:
{
tags: ["existingTag1", "existingTag2", ["one", "two", "etc"]]
}
So, tags[2] would be ["one", "two", "etc"], not what you want.
I have shown .reduce() which is an accumulator, which is a fancy, immutable way of doing this:
let updateTags = []
addedTags.forEach((tag) => {
updateTags.push(articles.updateOne({ slug: article.slug }, { $push: { tags: tag } }))
})
At this point, updateTags contains an array of functions, so calling Promise.all(updateTags) will run them all and detonate if any of them fail. Since we are using Mongo DB Native Driver, you will have to clean up if any errors occur, so you will probably want to track the pre-write state before calling Promise.all() (ie: What were the tags before?)
In the catch block, or catch block of the upper scope, you can "fire restore previous state" logic (rollback) and/or retry.
Something like:
// Upper scope
catch (e) {
if (e === 'addTags') rollback(previousState)
throw 'Problem occurred adding tags, please restart your computer.'
// This can now bubble up to your front-end client
}