When i creating indexes via createIndex or ensureIndex methods my node.js app crashing without providing any details of occured error.
Also i have noticed that all my code works very well with my localhost mongodb until i use remote mongdb atlas replica set.
Node.js: 8.5.0
Mongodb: 3.6.4
Mongodb driver for Node.js: 3.0.8
Example code (but i have tried different variants of implementation, nothing helps):
db.collection('stars').ensureIndex({name: 1}, { unique: true });
I am already listening for uncaughtException and unhandledRejection events, but they dont fire on this crash.
What can i do to get details or get rid of this error?
Thanks for any help!
I updated my mongodb driver to version 3.1.0 and eventually got error message with description of reason that causes the problem. According to error message: "The field "retryWrites" is not valid for for an index specification". So, i think the problem was with this field that i used to add to my connection uri string. After removing this field i got rid of this problem.
The problem might be connected with this issue: NODE-1641. I've got similar problem and found a few solutions:
Locking driver version at 3.1.0 and adding useNewUrlParser: true in connection options.
In createIndex add field retryWrites: null, so in your case it will look like:
db.collection('stars').ensureIndex({name: 1}, { unique: true, retryWrites: null });
However you need to check if it won't turn off retryable writes for that collection.
Last option is to vote up that issue and hope that it will be resolved soon, and then update your driver.
Not sure if this will help, but I'm currently trying to work around this.
If you remove {unique: true} it should get the job done, but it will make duplicate documents if you add the same document over again.
Related
Hi Im running couchabse server 7.1.3 docker container and trying to use the 3.3.4 couchbase java sdk version. I am facing an issue while performing subdocument mutation using the upsert command.
The issue I am facing is when I use the collection.mutateIn() method to perform a subdocument mutation using the upsert command, I am expecting to get the result of the operation as a MutateInResult object. when I try to use the MutateInResult.contentAs(0, String.class) to see the response I get the Index 0 is invalid error. I see that the operation as in is successful in the DB, I can see the desired json path has the updated value.
P.S: I only have a single mutation to perform which is the upsert command.
can someone please help if I am missing something here?
I'm trying to use MongoDB's Client-Side Filed Level Encryption feature with the community edition. I'm not interested in the auto-encryption feature. However, we need the auto-decryption feature which as per the docs is possible in the community edition as well.
We generally use mongoose in our application but I tried with native nodejs driver as well. Here's the code I'm using to create the connection. This works fine if I comment out the autoEncryption object. Doing so allows me to encrypt manually, but this way we will also have to decrypt manually, which kind of beats the purpose.
Some docs suggest adding bypassAutoEncryption: true with extraOptions object to the autoEncryption object. I've treid that as well as seen below.
const secureClient = new MongoClient('mongodb://someUri', {
useNewUrlParser: true,
useUnifiedTopology: true,
autoEncryption: {
keyVaultNamespace,
kmsProviders,
bypassAutoEncryption: true,
extraOptions: {
// mongocryptdBypassSpawn: true,
mongocryptdSpawnArgs: [ "--pidfilepath=bypass-spawning-mongocryptd.pid", "--port", "27021"],
mongocryptdURI: "mongodb://localhost:27021/db?serverSelectionTimeoutMS=1000"
},
}
});
My code is working till generating the master key, data-key and explicitly encrypting the data. Unfortunately, I haven't been able to set up the auto-decryption. To configure the client with CSFLE options the autoEncryption has to be passed in the options.
But whenever I pass this option, I get the following exception
(node:53721) UnhandledPromiseRejectionWarning: MongoServerSelectionError: connect ECONNREFUSED 127.0.0.1:27021
at Timeout._onTimeout (/Users/NiccsJ/ORI/code/testmongoEncryption/node_modules/mongodb/lib/sdam/topology.js:325:38)
at listOnTimeout (internal/timers.js:554:17)
at processTimers (internal/timers.js:497:7)
(Use `node --trace-warnings ...` to show where the warning was created)
I've followed almost all suggestions from the below refs. Surprisingly, mondodb-nodejs documentation doesn't even mention bypassAutoEncryption. I just happen to stumble across mongodb-c(point 3 & 4 below) driver documentation where I first found ant reference of such an option
https://github.com/mongodb/node-mongodb-native/blob/4ecaa37f72040ed8ace6eebc861b43ee9cb32a99/test/spec/client-side-encryption/tests/README.rst
https://github.com/Automattic/mongoose/issues/8167
http://mongocxx.org/mongocxx-v3/client-side-encryption/
https://mongodb.github.io/mongo-csharp-driver/2.11/reference/driver/crud/client_side_encryption/#explicit-encryption-and-auto-decryption
I was able to configure mongoShell with auto-decryption, meaning that my initial setup is not at fault. Also, it leads me to believe that there has to be a way to do it .via code as well.
My stack:
nodeJS: > 14.7
mongoDB: 4.4
OS: macOS for dev, prod will be on AmazonLinux2
Drivers: mongoose, native-nodejs, mongodb-client-encryption
It's not clearly mentioned in the docs. But from what I've read, automatic decryption doesn't require the enterprise-only mongocryptd process.
As mentioned in the official mongoDB-c-driver
Although automatic encryption requires MongoDB 4.2 enterprise or a MongoDB 4.2 Atlas cluster, automatic decryption is supported for all users. To configure automatic decryption without automatic encryption, set bypass_auto_encryption=True in the options::auto_encryption class.
I believe the bypassAutoEncryption option was made for this very purpose.
Not exactly an answer, but this is the best resolution at the moment.
I reported this as a bug on the official JIRA.
Turns out, this apparently is a bug with the node-mongo-native library.
As per their comment, this should be fixed in the next release.
I have a script that periodically runs aggregation on a mongodb collection. As the dataset has grown, the amount of time it takes to aggregate has also grown. My aggregation script has recently stopped working consistently, and the error logs show:
error: { [MongoError: server <x> timed out]
name: 'MongoError',
message: 'server <x> timed out' }
I've tried debugging this, and the only pattern I can find is that this timeout seems to only occur when the aggregation takes longer than 2 minutes (it times out right around 2m). Does anyone have additional debugging tips for this? The 2-minute thing is giving me the impression that I just need to configure some timeout somewhere but I can't figure out where or if i'm just falling into a red-herring trap.
About the system configuration: This aggregation script is a node.js (v5.9.1) application running in an alpine-based docker (v1.9.1) container. It uses the mongodb node driver (v2.1.19). Single mongodb server (though this is also happening in a separate environment with a replSet) running mongod (v3.2.6)
I got the same problem for logs time aggregation. I think I have the solution for you.
I found that the option socketTimeoutMS is responsible for that.
Check your mongo_client.js default socketTimeoutMS value. For me it was 2min. Mongodb module version 2.1.18.
So just add this option into your url :
mongodb://localhost:27017/test?maxPoolSize=2&socketTimeoutMS=60000
It will set timeout to 10 mins. That does the trick for me.
I wrote a node web app and created a mongoDb database on my local system. I was using the following code to connect to local mongodb from node js
var mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/db_name'); //local
And everything was working fine on my local machine. So I went on and created an mlab account and created a database. But when I tried to run the code by changing the connection string, connections are still established I believe. But the find and save requests are not invoking the callbacks, even no errors shows up. All requests are getting timed out.
var mongoose = require('mongoose');
mongoose.connect("mongodb://user:pass#ds036789.mlab.com:36789/db_name"); //mlab
Another thing I noticed is that I cannot ping ds036789.mlab.com. But TCP connections are succeeding when I tried the nc command
nc -w 3 -v ds036789.mlab.com 36789
I even tried deploying to azure. Which doesn't work either. Any help is much appreciated. Thanks.
EDIT:
Not being able to ping was due to the fact that I used azure hosting. It is expected. And I also found out that I get this error while trying to connect :
connection error: { [MongoError: auth failed] name: 'MongoError', ok: 0, errmsg: 'auth failed', code: 18 }
Credentials are correct though.
From the error mesasge it seems like you are using invalid auth details
This is most likely happen when you do not create username and password for individual database i.e, db_name in you case.
Check mLabs account and create username and password for db_name database and update your connection string.
According to the error information, as #Astro said, it seems to be caused by using invalid auth user/password which be created for database.
Did you create a new user for connecting the database, not account user for mlab? Such as the figures below shown.
Fig 1. A database user is required for connecting
Fig 2. Users list for the database
Hope it helps.
I figured out the issue, it wasn't an issue with the credentials. It was an issue with the mongoose version. The mongoose version I used didn't support the authentication. I had to remove the package and reinstall the latest version. with
node install mongoose#latest
Hope it helps someone. And thanks for the answers :)
I'm creating a node.js app that sends reminders using agenda.js. It works perfectly when I test it locally, but when I test it on OpenShift, I get the following error message:
MongoError: No such cmd: createIndexes
I only get this error when the information for a new reminder is sent to the server, i.e. only when agenda.js is used.
I've looked up createIndexes, and it seems that it was implemented in version 2.6 of MongoDB, and OpenShift currently only appears to support version 2.4.
My question is, is there a way around this? Perhaps a way to manually upgrade to the latest version of MongoDB, or not to use a cartridge at all (not sure what that actually is)?
Before 2.6, there wasn't an internal command called CreateIndexes. It was necessary to insert and object on the system.indexes collection directly.
On mongo shell, there were 2 helpers for that, with different names:
db.collection.createIndex, which still exists nowadays;
db.collection.ensureIndex, which was removed from 2.6 on.
I couldn't understand what exactly is issuing the create index command. Is it your SDK? Because it is supposed to be done just once, and not on every insert.
I had the same issue, and, as a workaround, used Agenda 0.6.28 instead (as it worked in a previous project of mine under that version).
Mind that Agenda does not emit a 'ready' event in this lower version, so you'd better call the 'start' function directly:
agenda.define('delete old session logs', jobs.cleanSessionLogs);
agenda.on('fail', function(err, job) {
errorLog.error({ err: err, job: job });
});
agenda.every('1 hour', 'delete old session logs');
agenda.start();
instead of:
agenda.define('delete old session logs', jobs.cleanSessionLogs);
agenda.on('fail', function(err, job) {
errorLog.error({ err: err, job: job });
});
agenda.on('ready', function() {
agenda.every('1 hour', 'delete old session logs');
agenda.start();
});