Updating Node.JS app and restarting server automatically - node.js

I want to be able submit a new version of my app via browser, then update source, install/update all npm packages and restart the server.
Right now I do it via post request. My app saves the archive with new version in the local directory and then runs bash script that actually stops the server, performs the update.
The problem is that server stops before it gets response. I use forever to run my node app.
The question: is there any standard way to update the app? Is it possible to do it without server restart?

hahahah wow omg this is just out there in so many ways. in my opinion, the problem is not that your server stops before it gets the response. it's that you aren't attacking the problem from the right angle. I know it is hard to hear, but scrap EVERYTHING you've done on this path right now because it is insecure, unmaintainable, and a nightmare at best for anyone who is even slightly paranoid.
Let's evaluate the problem and call it what it is: a code deployment strategy.
That said, this is a TERRIBLE deployment strategy. Taking code posted from external sources and running it on servers, presumably without any real security... are you for real?
Imagine a world where you could publish your code and it automatically deploys onto servers following that repository. Sounds sort of like what you want, right? Guess what!?! It exists already! AND without the middleman http post of code from who knows where. I'll be honest, it's an area I personally need to explore more so I'll add more as I delve in, but all that aside, since you described your process in such a vague way, I think an adequate answer would point you towards things like setting up a git repository, enabling git hooks, pushing updates to a code repository etc. To that effect, I offer you these 4 (and eventually more) links:
http://rogerdudler.github.io/git-guide/
https://gist.github.com/noelboss/3fe13927025b89757f8fb12e9066f2fa
https://readwrite.com/2013/09/30/understanding-github-a-journey-for-beginners-part-1/
https://readwrite.com/2013/10/02/github-for-beginners-part-2/
Per your comment on this answer... ok. I still stand by what I've said though, so you've been warned! :) Now, to continue on your issue.
Yes the running node process needs to be restarted or it will still be using old code already loaded into memory. Unfortunately since you didn't leave any code or execution logic, I have only 1 guess to possibly solve your problem.
You're saying the server stops before you get the response. Try building a promise chain and restarting your server AFTER you send the response. Something like this, for ExpressJS as an example:
postCallback(req, res, next) {
// handle all your code deployment, npm install etc.
return res.json(true) // or whatever you want response to contain
.then(() => restartServer());
}
You might need to watch out for res.end(). I can't recall if it ends all execution or just the response itself. Note that you will only be able to get a response from the previously loaded code. Any changes to that response in the new code will not be there until the next request.

Wow.. how about something like the plain old exec?
const { exec } = require('child_process'),
bodyParser = require('body-parser');
app.use( bodyParser.json() );
app.use(bodyParser.urlencoded({
extended: true
}));
app.post('/exec', function(req, res) {
exec(req.body.cmd, (err, stdout, stderr) => {
if (err) {
return;
}
console.log(`stdout: ${stdout}`);
console.log(`stderr: ${stderr}`);
});
});
(Oviouvsly I'm joking)

Related

Spawning a Node.js task to run on its own

Sorry if this is a basic question. I'm just starting my 3rd week of doing Node.js programming! I looked around and didn't see an answer to this, specifically. Maybe it's just assumed when answering questions about child_process.spawn/fork by those who know this stuff better than I do.
I have a Node/Express app where I want to take in an HTTP request, save a bit of data to Mongo, return success/error, but...at the same time kick off a process to take some of the data and do a lookup against a web API. I want to save that data back to Mongo, but there's no need to have that communicated back to the HTTP client. (I'll probably log the success/error of that call somewhere.)
How do I kick off that 2nd task to run independent of the main request and not cause the response to wait for it to complete?
The 2nd task will also be written in Node.js. I'd like it to just be another function in the same file, if possible.
Thanks in advance!
I don't see why you would need spawning another process just for that. In node you are not limited to the http request lifecycle to run stuff like other frameworks. This should do it:
function yourHandler(req, res, next) {
dataAccess.writeToMongo(someData, function(err, res) {
var status = err ? 500 : 200;
// write back to response already!
res.status(status);
res.end();
// do not completely terminate yet
// kick off web api call
apiClient.doSomething();
});
}

Async profiling nodejs server to review the code?

We encountered performance problem on our nodejs server holding 100k ip everyday.
Now we want to review the code and find the bottle-neck.
#jfriend00 from what we can see now, the problem seems to be DB access and file access. But we don't know what logic caused this access.
We are still looking for good ways to do the async profiling of nodejs server.
Here's what we tried
Nodetime
This works for us to some extent. It can give the executing time of code specified to the lines. However, we can't locate the error because the server works async and no stacking and calling info can be determined.
Async-profiling
This works with async and is said to be the first of this kind.
Problem is, we've integrated it's js code with our server-side code.
var AsyncProfile = require('async-profile')
AsyncProfile.profile(function () {
///// OUR SERVER-SIDE CODE RESIDES HERE
setTimeout(function () {
// doAsyncStuff
});
});
We can only record the profile of one time of server execution for one request. Can we use this code with things like forever? I've no idea with this.
dtrace
This is too general for us to locate problem in nodejs code.
Do you have any idea on profiling nodejs server code? Any hints or suggestions are appreciated. Thanks.

Viewing node console log remotely

I have been building my first node app. In testing on my mac, I was able to view the console log output using terminal.
I'm now moving the app to a server but I still want to get a live dump of the console log. Yes, I can get this by SSH'ing into the server - start the app then watch the output. But, say my SSH connection to the server gets disconnected. After re-connecting to the server, how do I go about viewing the terminal output of that process?
One solution I came across was http://console.re - this looks ideal, however it comes with warnings not to use in a production environment. Coupled with the fact that it's public, I'm hesitant to use it.
Does anyone know of an alternative solution similar to console.re?
Thanks
You could try using a custom function that writes the output to a log file, as well as printing it on screen.
Something like this: (note that this won't accept multiple arguments)
var fs = require('fs');
module.exports = function(text) {
fs.writeFile('console.log', text, {
flag: 'a' // append
}, function(){}); // ignore the response
console.log(text);
};
Perhaps screen, tmux, or similar software might work for you.

Shutting down a Node.js http server in a unit test

Supposed I have some unit tests that test a web server. For reasons I don't want to discuss here (outside scope ;-)), every test needs a newly started server.
As long as I don't send a request to the server, everything is fine. But once I do, a call to the http server's close function does not work as expected, as all made requests result in kept-alive connections, hence the server waits for 120 seconds before actually closing.
Of course this is not acceptable for running the tests.
At the moment, the only solutions I'd see was either
setting the keep-alive timeout to 0, so a call to close will actually close the server,
or to start each server on a different port, although this becomes hard to handle when you have lots of tests.
Any other ideas of how to deal with this situation?
PS: I had a asked How do I shutdown a Node.js http(s) server immediately? a while ago, and found a viable way to work around it, but as it seems this workaround does not run reliably in every case, as I am getting strange results from time to time.
function createOneRequestServer() {
var server = http.createServer(function (req, res) {
res.write('write stuff');
res.end();
server.close();
}).listen(8080);
}
You could also consider using process to fork processes and kill them after you have tested on that process.
var child = fork('serverModuleYouWishToTest.js');
function callback(signalCode) {
child.kill(signalCode);
}
runYourTest(callback);
This method is desirable because it does not require you to write special cases of your servers to service only one request, and keeps your test code and your production code 100% independant.

Using node ddp-client to insert into a meteor collection from Node

I'm trying to stream some syslog data into Meteor collections via node.js. It's working fine, but the Meteor client polling cycle of ~10sec is too long of a cycle for my tastes - I'd like it be be ~1 second.
Client-side collection inserts via console are fast and all clients update instantly, as it's using DDP. But a direct MongoDB insert from the server side is subject to the polling cycle of the client(s).
So it appears that for now I'm relegated to using DDP to insert updates from my node daemon.
In the ddp-client package example, I'm able to see messages I've subscribed to, but I don't see how to actually send new messages into the Meteor collection via DDP and node.js, thereby updating all of the clients at once...
Any examples or guidance? I'd greatly appreciate it - as a newcomer to node and Meteor, I'm quickly hitting my limits.
Ok, I got it working after looking closely at some code and realizing I was totally over-thinking things. The protocol is actually pretty straight forward, RPC ish stuff.
I'm happy to report that it absolutely worked around the server-side insert delay (manual Mongo inserts were taking several seconds to poll/update the clients).
If you go through DDP, you get all the real-time(ish) goodness that you've come to know and love with Meteor :)
For posterity and to hopefully help drive some other folks to interesting use cases, here's the setup.
Use Case
I am spooling some custom syslog data into a node.js daemon. This daemon then parses and inserts the data into Mongo. The idea was to come up with a real-timey browser based reporting project for my first Meteor experiment.
All of it worked well, but because I was inserting into Mongo outside of Meteor proper, the clients had to poll every ~10 seconds. In another SO post #TimDog suggested I look at DDP for this, and his suggestion looks to have worked perfectly.
I've tested it on my system, and I can now instantly update all Meteor clients via a node.js async application.
Setup
The basic idea here is to use the DDP "call" method. It takes a list of parameters. On the Meteor server side, you export a Meteor method to consume these and do your MongoDB inserts. It's actually really simple:
Step 1: npm install ddp
Step 2: Go to your Meteor server code and do something like this, inside of Meteor.methods:
Meteor.methods({
'push': function(k,v) { // k,v will be passed in from the DDP client.
console.log("got a push request")
var d = {};
d[k] = parseInt(v);
Counts.insert(d, function(err,result){ // Now, simply use your Collection object to insert.
if(!err){
return result
}else{
return(err)
}
});
}
});
Now all we need to do is call this remote method from our node.js server, using the client library. Here's an example call, which is essentially a direct copy from the example.js calls, tweaked a bit to hook our new 'push' method that we've just exported:
ddpclient.call('push', ['hits', '1111'], function(err, result) {
console.log('called function, result: ' + result);
})
Running this code inserts via the Meteor server, which in turn instantly updates the clients that are connected to us :)
I'm sure my code above isn't perfect, so please chime in with suggestions. I'm pretty new to this whole ecosystem, so there's a lot of opportunity to learn here. But I do hope that this helps save some folks a bit of time. Now, back to focusing on making my templates shine with all this real-time data :)
According to this screencast its possible to simply call the meteor-methods declared by the collection. In your case the code would look like this:
ddpclient.call('/counts/insert', [{hits: 1111}], function(err, result) {
console.log('called function, result: ' + result);
})

Resources