I'm using Mongo Atlas with Prisma. Whenever I launch my project, it takes about 3 minutes for database to connect, and in this period of time, I can't see anything in Mongo Atlas' connection tab. I've tried Prisma's logger, no error prints at all. Interestingly, it connects but it just takes time.
It eventually connects, but this issue with connection time makes using hot reload quite hard. I've tried allowing incoming Node apps in Advanced Firewall Settings, and my firewall is disabled. Actually I couldn't find a lot of people experiencing this, so I couldn't try much to be honest. And to note, when somebody else tries to launch exact same code, their database starts working instantly.
Related
I have a nodejs/ExpressJS/Mongodb/Mongoose app hosted on aws elasticbeanstalk.
The problem is elasticbeanstalk health degrades randomly ( no specific times ), that happens because any request that requires database interaction results the following in logs:
*1360931 upstream timed out (110: Connection timed out) while reading response header from upstream
This happens no matter how much data I try to load. it happens with the least amount of data, this can last from a minute up to 20 minutes and it works again on its own, it is completely random.
And I can force it to work immediately by restarting the environment ( I connect to mongodb using connection string on app startup ).
While other requests that don't require database interaction work 100%.
The thing is while database queries aren't working , I can connect to the same database from localhost and database requests work like a charm, they even work really fast.
What is even more strange is I have 4 other identical apps with the same setup, and This situation doesn't occur with any of them, only this app faces this problem !
What is the problem here ?
The above error usually means that your server closed the connection due to shorter timeout , but your application is not aware about it. You may need to check your connection string and modify the timeouts maybe decrease them , example connection string:
MONGO_URI=mongodb://user:password#127.0.0.1:27017/dbname?keepAlive=true&poolSize=30&autoReconnect=true&socketTimeoutMS=360000&connectTimeoutMS=360000
I've been working with MongoDB for a while now and I've been liking it a lot. One thing I do not understand however is "Connections". I've searched online and everything just has very vague and basic answers. I'm using MongoDBs cloud service called "Atlas" and it describes the connection count as
The number of currently active connections to this server. A stack is allocated per connection; thus very many connections can result in significant RAM usage.
However I have a few questions.
What is a connection I guess? As I understand it, a connection is made between the server and the database service. Essentially when I use mongoose.connect(...);, a connection is made. So at most, there should only be one connection. However when I was testing my program I noticed my connection count was at 2 and in some moments it spiked up all the way to 7 and went to 5 and fluctuated. Does a "connection" have anything to do with the client? On the dashboard of Atlas it says I have a max connection amount of 500. What does this value represent? Does this mean only 500 users can use my website at once? If that's the case, how can I increase that number? Or how can I make sure that more than 500 connections never get passed? Or is a connection something that gets opened and I have to manually close myself? Because I've been learning from tutorials and I've never seen/heard anything like that.
Thanks!
mongoose.connect doesn't limit itself to 1 connection to the Mongo Server.
By default, mongoose creates a pool of 5 connections to Mongo.
You can change this default if necessary.
mongoose
.connect(mongoURI, {poolSize : 200});
See https://mongoosejs.com/docs/connections.html
More number of connections which you see in Atlas because there are some internal connections are also made in order to make the cluster running, these may include the connections from:
Connections made from a client.
Internal connections between primary and secondaries.
As it is a hosted service and everything is being monitored so connections from the monitoring agent.
As automation works, so the connections from the automation agent as well.
Hence whenever a new cluster is being created in Atlas, you will always see some connections in the metrics Page even though no client is being connected.
we are using https://github.com/oracle/node-oracledb to connect to Oracle DB with nodejs, oracle client library 18.0. Everything works fine till a few hours later, the browser shows "Server does not respond". It is quite similar to the issue at https://github.com/oracle/node-oracledb/issues/725, where the blocking on getConnection() happens. We tried poolPingIntervalSetting and it was not working. Any idea?
The most common issue is not having enough threads. See https://oracle.github.io/node-oracledb/doc/api.html#numberofthreads If the pool grows due to load but you don't have enough threads, you may lock up.
Then make sure your application is releasing connections back to the pool, including in error situations. Use pool._logStats() and see if you are leaking connections.
Review the node-oracledb doc on connection pool best practices https://oracle.github.io/node-oracledb/doc/api.html#conpoolsizing
Look at network issues and make sure firewalls etc aren't killing sessions. Open an issue on GitHub to take this further.
Essentially, I'm trying to work out the best way to ensure that a user is connected to the server / the internet and is thus able to make requests in my application without error.
I have come across various solutions, but I can;t really decide what is the best performing or useful.
Websockets, using Socket.io to keep an open connection with the server for each client. Also opens up the possibility for real time updates in my app, which could be a nice thing in the future. However, having lots of open sockets is sure to be hard hitting in performance.
Polling, so having an endpoint in my API that the angular app hits every 5 seconds or so to check the user is connected. Again, seems like it isn't a good idea to be hitting the server a load.
Waiting for an error, then start polling every couple of seconds to wait for the connection to be re-established. This is a little change on the above. However, you are still waiting for a user to fail, which isn't good for user experience.
Does anybody have any informed input on this issue?
Thanks
I am using MongoDB, so I am connecting trough MongoClient.connect
but I have to use that, for every route where I want to work with the database.
Tried to preload it to an object, but then the changes are not visible, till the server is restarted. Right now, it's working properly, I am only a bit worried about the performance.
Is there a better way to do that?
THe correct answer is "it depends".
For a simple desktop application where your NodeJS program is the only client: sure. A persistent connection is fine.
For an enterprise application with 100s or 1000s of concurrent users each connecting independently: no, you probably do NOT want to hold the connection open "forever".
One possible solution for the latter scenario is Connection Pooling.
You should only need to connect to your DB once - when the server starts. As long as the server is running, the connection should persist. There is no reason to connect multiple times.