I am using an Express server in NodeJS v14.15.1 to handle HTTP GET and POST requests. The server performs some cryptographic operations and obtains a key which must be used for subsequent requests. The obtained key is set as a global variable within my index.js file (where my express() app resides). However, the server restarts automatically (I am using nodemon) upon handling each HTTP request, and in doing so it erases the key global variable. So the next request which relies on reading the global key variable is unable to succeed. NB: The key cannot be stored on-disk or on the client-side due to security reasons. Also, this is for a university assignment, not a real production environment.
How can I keep the global variable upon server restart?
Any help is greatly appreciated.
None of the nodejs/Express environment automatically survives from one restart to the next. So, if you have specific data that you want to always be available after a restart, then you would typically save it to disk (often in a JSON file) every time it changes and then every time your server starts, it can read that state from the previous JSON file and initialize your variables from that data.
However, the server restarts automatically (I am using nodemon) upon handling each HTTP request
This should not happen, so your first thing to solve it to stop the server from restarting. Your server should run for days or weeks and be able to field millions of http requests without restarting. Is the server crashing and restarting or is nodemon seeing something change and automatically killing/restarting your server?
Nodemon is sometimes used in a "developer" mode such that it automatically restarts your server anytime files in a specific directory are changed. This can facilitate faster development cycles if you are editing your source files. But, this should NOT be happening except when you edit your server source files and only when you are using nodemon in a "debug" or "development" mode. If this is why nodemon is restarting your server, then you probably need to tweak the configuration so it isn't detecting file changes that are part of your normal server operation and thus doesn't restart just because your server does something normal.
The key cannot be stored on-disk
Well, there's no simple way to get data to survive a server restart without storing the data somewhere. NOTHING from your Express process survives a restart so you can't only keep it in the Express process if you want to have access to it again after a restart. So, it appears to me that you've put yourself in a box.
Your options are to either stop the server from restarting in the first place or find a secure place to store the key.
Related
After running node app.js in the OS X Terminal to start the server, any file that has its contents altered in the editor and then saved will have the changes immediately reflected when I refresh the localhost:3000 page, even when the server has not been restarted (so far, I have tried this on .ejs, .css and .js files).
Details:
Problem only happens in one directory; I am unable to replicate it in another similarly structured sibling directory.
I do not have Nodemon/Forever/Node Supervisor installed, as far as I know.
I tried killall node (I thought that I had nothing to lose as this should have been the only process running on my computer; I now realise that this may have been the wrong approach)
I have already tried restarting the computer but to no avail; the problem still persists.
When I do stop the application, the localhost:3000 page shows that "This site can’t be reached", i.e. the server can and has been stopped.
Why do I not need to stop (Ctrl+C) then restart (⇧ and Enter) the application every time I save a file in the editor? What is going on here and how I could replicate it or make it stop?
HTML,CSS and frontend JS modifications do not require you to restart your node process as they are read on every request, unlike your server side code that is loaded in memory as soon as you start up your server.
Basically what happens is that when you run your server side JS like app.js it loads in memory and any code in that file will stay in memory as long as the process is running, when you serve a html file your framework reads the html file every time a request is received, due to which changes to html file are reflected every-time without you having to restart your server , as well as JS changes in your html file are reflected as they are not part of server side process. On every request your process opens the html file reads its content and sends to browser using HTTP protocol and your browser reads your html response and request your server for necessary JS files included in your html which are in your public folder.Hope this helps.
Consider this scenario:
Socket.io app went down (or restarted) for some reason and took about 2 seconds before it started again (considering the use of production manager app ie: PM2).
Within the 3 second down time a client tried to request the client socket.io.js script (localhost:xxxx/socket.io/socket.io.js) and resulted as a failed request (error 500, 404, or net::ERR_CONNECTION_REFUSED) before the server got started again.
After the three second downtime the server file is available again.
So now i have no other way but to inform the user to refresh to resume real time transactions.
I cannot retry to reconnect to the socket.io server because i do not have the client script.
But if it is served somewhere else, perhaps at the same dir where jQuery is, i could just listen if io is available again by writing a simple retry function that fires for every few seconds.
In general, it's a good idea to use the version served by Socket.IO, as you'll have guaranteed compatibility. However, as long as you stay on top of making sure you deploy the right versions, it's perfectly fine to host that file somewhere else. In fact, it's even preferred since you're taking the static load off your application servers and putting it elsewhere.
An easy way to do what you want is to configure Nginx or similar to cache that file and serve a stale copy when the upstream server (your Node.js with Socket.IO server) is down. https://serverfault.com/q/357541/52951
User case:
My nodejs server start with a configuration wizard that allow user to change the port and scheme. Even more, update the express routes
Question:
Is it possible to apply the such kind of configuration changes on the fly? restart the server can definitely bring all the changes online but i'm not sure how to trigger it from code.
Changing core configuration on the fly is rarely practiced. Node.js and most http frameworks do not support it neither at this point.
Modifying configuration and then restarting the server is completley valid solution and I suggest you to use it.
To restart server programatically you have to execute logics outside of the node.js, so that this process can continue once node.js process is killed. Granted you are running node.js server on Linux, the Bash script sounds like the best tool available for you.
Implementation will look something like this:
Client presses a switch somewhere on your site powered by node.js
Node.js then executes some JavaScript code which instructs your OS to execute some bash script, lets say it is script.sh
script.sh restarts node.js
Done
If any of the steps is difficult, ask about it. Though step 1 is something you are likely handling yourself already.
I know this question was asked a long time ago but since I ran into this problem I will share what I ended up doing.
For my problem I needed to restart the server since the user is allowed to change the port on their website. What I ended up doing is wrapping the whole server creation (https.createServer/server.listen) into a function called startServer(port). I would call this function at the end of the file with a default port. The user would change port by accessing endpoint /changePort?port=3000. That endpoint would call another function called restartServer(server,res,port) which would then call the startServer(port) with the new port then redirect user to that new site with the new port.
Much better than restarting the whole nodejs process.
I'm working with Node.js on a serious project for the first time, building a multiplayer card game. I'm using socket.io for websockets. The Node.js server should run 24/7 once the game is live.
Currently, I have a few issues I need to iron out. For clarity, I will split them into two sub-questions below.
First, every time there's an error the whole server crashes. I will try to iron out all bugs before going live of course, but it would still be nice if we could avoid killing the whole server process when one card game's data is corrupted. For example, if I refer to a non-existing element of an object, of course Node throws an error. That's fine, but is there a way to prevent such occurrences from crashing the server (besides making sure that this type of thing doesn't happen in the first place)?
Second, game states are stored on the server during gameplay. So, if there are 3 games in progress, there will be states[1], states[2] and states[3] on the Node server. When the server crashes, I'd like to somehow keep hold of this data so the server could be restarted immediately and the data restored. I think sessions on the Node server are not an option because they will (presumably) die with the server, no? Storing everything to MySQL upon every game action seems like a huge waste of resources (but correct me if I'm wrong).
This all comes down to: how can I make sure the server can function autonomously through any potential problems without crashing and, if it has to crash, how can I make sure it doesn't take all ongoing games with it?
If bad state data is causing your app to crash, wouldn't reloading it after restart just cause another crash?
You should always validate that incoming data is completely valid before storing it, whether in-memory or in a database.
Whenever accessing a property of an object that may not exist, always check for that object's existence.
if (state[n].obj && state[n].obj.prop == val) ...
As far as storing your data out-of-process, your app sounds like a good candidate for a simple key-value store like Redis. If you store all your state data in Redis, it will survive node restarts, and performance will still be similar to in-proc memory. It also gives you the benefit of being able to scale beyond a single node process to handle heavy load.
I have an internal cherrypy server that serves static files and answers XMLRPC requests. All works fine, but 1-2 times a day i need to update this static files and database. Of course i can just stop server, run update and start server. But this is not very clean since all other code that communicate with server via XMLRPC will have disconnects and users will see "can't connect" in broswers. And this adds additional complexity - i need some external start / stop / update code, wile all updaes can be perfectly done within cherrypy server itself.
Is it possible to somehow "pause" cherrypy programmatically so it will server static "busy" page and i can update data without fear that right now someone is downloading file A from server and i will update file B he wants next, so he will get different file versions.
I have tried to implement this programmatically, but where is a problem here. Cherrypy is multithread (and this is good), so even if i introduce a global "busy" flag i need some way to wait for all threads to complete aready existing tasks before i can update data. Can't find such way :(.
CherryPy's engine controls such things. When you call engine.stop(), the HTTP server shuts down, but first it waits for existing requests to complete. This mode is designed to allow for debugging to occur while not serving requests. See this state machine diagram. Note that stop is not the same as exit, which really stops everything and exits the process.
You could call stop, then manually start up an HTTP server again with a different app to serve a "busy" page, then make your edits, then stop the interim server, then call engine.start() and engine.block() again and be on your way. Note that this will mean a certain amount of downtime as the current requests finish and the new HTTP server takes over listening on the socket, but that will guarantee all current requests are done before you start making changes.
Alternately, you could write a bit of WSGI middleware which usually passes requests through unchanged, but when tripped returns a "busy" page. Current requests would still be allowed to complete, so there might be a period in which you're not sure if your edits will affect requests that are in progress. How to write WSGI middleware doesn't fit very well in an SO reply; search for resources like this one. When you're ready to hook it up in CherryPy, see http://docs.cherrypy.org/dev/concepts/config.html#wsgi