Does Meteor have a setting to timeout and retry if its MongoDB does not give a response in x seconds? Wondering if anyone has tried this.
I am interested in running a MongoDB database remote to the Meteor production app. The Meteor-to-Mongo connection will be quick, just 3-9 milliseconds away, but I also want to understand how Meteor (and NodeJS) would react to a brief network outage. Would the app hang while waiting for a long timeout period? How can I force a 1 second timeout/retry to avoid a hang?
You can specify timeout in the mongo URL:
MONGO_URL=mongodb://host:port/db?connectTimeoutMS=60000&socketTimeoutMS=60000
but let's say you have a network outage, what does a short timeout give you?
your app will hang anyways...
To get high availability, look into replica sets.
https://docs.mongodb.com/manual/tutorial/deploy-replica-set/
Related
I have a nodejs/ExpressJS/Mongodb/Mongoose app hosted on aws elasticbeanstalk.
The problem is elasticbeanstalk health degrades randomly ( no specific times ), that happens because any request that requires database interaction results the following in logs:
*1360931 upstream timed out (110: Connection timed out) while reading response header from upstream
This happens no matter how much data I try to load. it happens with the least amount of data, this can last from a minute up to 20 minutes and it works again on its own, it is completely random.
And I can force it to work immediately by restarting the environment ( I connect to mongodb using connection string on app startup ).
While other requests that don't require database interaction work 100%.
The thing is while database queries aren't working , I can connect to the same database from localhost and database requests work like a charm, they even work really fast.
What is even more strange is I have 4 other identical apps with the same setup, and This situation doesn't occur with any of them, only this app faces this problem !
What is the problem here ?
The above error usually means that your server closed the connection due to shorter timeout , but your application is not aware about it. You may need to check your connection string and modify the timeouts maybe decrease them , example connection string:
MONGO_URI=mongodb://user:password#127.0.0.1:27017/dbname?keepAlive=true&poolSize=30&autoReconnect=true&socketTimeoutMS=360000&connectTimeoutMS=360000
I'm using Mongo Atlas with Prisma. Whenever I launch my project, it takes about 3 minutes for database to connect, and in this period of time, I can't see anything in Mongo Atlas' connection tab. I've tried Prisma's logger, no error prints at all. Interestingly, it connects but it just takes time.
It eventually connects, but this issue with connection time makes using hot reload quite hard. I've tried allowing incoming Node apps in Advanced Firewall Settings, and my firewall is disabled. Actually I couldn't find a lot of people experiencing this, so I couldn't try much to be honest. And to note, when somebody else tries to launch exact same code, their database starts working instantly.
I've been working with MongoDB for a while now and I've been liking it a lot. One thing I do not understand however is "Connections". I've searched online and everything just has very vague and basic answers. I'm using MongoDBs cloud service called "Atlas" and it describes the connection count as
The number of currently active connections to this server. A stack is allocated per connection; thus very many connections can result in significant RAM usage.
However I have a few questions.
What is a connection I guess? As I understand it, a connection is made between the server and the database service. Essentially when I use mongoose.connect(...);, a connection is made. So at most, there should only be one connection. However when I was testing my program I noticed my connection count was at 2 and in some moments it spiked up all the way to 7 and went to 5 and fluctuated. Does a "connection" have anything to do with the client? On the dashboard of Atlas it says I have a max connection amount of 500. What does this value represent? Does this mean only 500 users can use my website at once? If that's the case, how can I increase that number? Or how can I make sure that more than 500 connections never get passed? Or is a connection something that gets opened and I have to manually close myself? Because I've been learning from tutorials and I've never seen/heard anything like that.
Thanks!
mongoose.connect doesn't limit itself to 1 connection to the Mongo Server.
By default, mongoose creates a pool of 5 connections to Mongo.
You can change this default if necessary.
mongoose
.connect(mongoURI, {poolSize : 200});
See https://mongoosejs.com/docs/connections.html
More number of connections which you see in Atlas because there are some internal connections are also made in order to make the cluster running, these may include the connections from:
Connections made from a client.
Internal connections between primary and secondaries.
As it is a hosted service and everything is being monitored so connections from the monitoring agent.
As automation works, so the connections from the automation agent as well.
Hence whenever a new cluster is being created in Atlas, you will always see some connections in the metrics Page even though no client is being connected.
We're running Node (v0.10.38) with Express (4.0.0), proxied through nginx (1.2.1), which usually works great. Recently, however, we switched to a new server setup. Now, roughly 30 minutes after starting up the server, the server starts returning 504s (Gateway Timeout). Accessing Node directly from the server (bypassing nginx) also times out. Every so often, we got a series of ETIMEDOUT errors from redis, but connecting to the redis server from the server works from the command line. Furthermore, the server started returning 504s even before redis errors came up anyways. Anyways, after updating our redis middleware (connect-redis) to the newest version, these errors stopped, but the 504s still occurred. However, after disabling the connection to redis in our code for 10 hours, no 504 occurred. We've tried sending a redis ping periodically to prevent the error, believing that to be the cause, but 504s continue. When not connecting to redis, the server doesn't 504, so it is likely tied to redis in some way. Anything else we can try?
Sorry if there's not much to work with. We don't have that much either, and are eager to solve this issue as soon as possible. If there's any more specifics needed, I can update the question. Thank you.
Still don't know the root cause, but we ended up fixing this by pinging Redis every minute so that the connection wouldn't get killed.
I am trying to scale a messaging app. Im using nodeJS with Socket.io and Redis-Store on the backend. The client can be iphone native browser, android browsers .. etc
I am using SSL for the node connection, using Nginx to load balance the socket connections. I am not clustering my socket.io app , instead i am load balancing over 10 node servers ( we have a massive amount of users ). Everything looks fine when the transport is Websockets , however when it falls back to xhr-polling ( in case of old android phones ) I see a HUGE response time of up to 3000 rpm in New-relic. And I have to restart my node servers every hour or so otherwise the server crashes.
I was wondering if I am doing anything wrong , and if there are any measures I can take to scale socket.io when using xhr-polling transport ? like increasing or decreasing the poll duration ?
You are not doing anything wrong, xhr-polling is also called long polling. The name comes from the fact that the connection is maintained open longer, usually until some answer can be sent down the wire. After the connection closes, a new connection is open waiting for the next piece of information.
You can read more on this here http://en.wikipedia.org/wiki/Push_technology#Long_polling
New Relic shows you the response time of the polling request. Socket.IO has a default "polling duration" of 20 seconds.
You will get a higher RPM for smaller polling duration and a smaller RPM for a higher polling duration. I would consider increasing the polling duration or just keep the 20 sec default.
Also, to keep New Relic from displaying irrelevant data for the long polling, you can add ignore rules in the newrelic.js that you require in you app. This is also detailed in the newrelic npm module documentation here https://www.npmjs.org/package/newrelic#rules-for-naming-and-ignoring-requests