child_process spawn Race condition possibility in nodejs - node.js

I'm starting to learn and use node and I like it but I'm not really sure how certain features work. Maybe you can help me resolve one such issue:
I want to spawn local scripts and programs from my node server upon rest commands. looking at the fs library I saw the example below of how to spawn a child process and add some pipes/event handlers on it.
var spawn = require('child_process').spawn,
ps = spawn('ps', ['ax']),
grep = spawn('grep', ['ssh']);
ps.stdout.on('data', function (data) {
grep.stdin.write(data);
});
ps.stderr.on('data', function (data) {
console.log('ps stderr: ' + data);
});
ps.on('close', function (code) {
if (code !== 0) {
console.log('ps process exited with code ' + code);
}
grep.stdin.end();
});
grep.stdout.on('data', function (data) {
console.log('' + data);
});
grep.stderr.on('data', function (data) {
console.log('grep stderr: ' + data);
});
grep.on('close', function (code) {
if (code !== 0) {
console.log('grep process exited with code ' + code);
}
});
What's weird to me is that I don't understand how I can be guaranteed that the event handler code will be registered before the program starts to run. It's not like there's a 'resume' function that you run to start up the child. Isn't this a race condition? Granted the condition would be minisculy small and would almost never hit because its such a short snipping of code afterward but still, if it is I'd rather not code it this way out of good habits.
So:
1) if it's not a race condition why?
2) if it is a race condition how could I write it the right way?
Thanks for your time!

Given the slight conflict and ambiguity in the accepted answer's comments, the sample and output below tells me two things:
The child process (referring to the node object returned by spawn) emits no events even though the real underlying process is live / executing.
The pipes for the IPC are setup before the child process is executed.
Both are obvious. The conflict is w.r.t. interpretation of the OP's question:-
Actually 'yes', this is the epitome of a data race condition if one needs to consider the real child process's side effects. But 'no', there's no data race as far as IPC pipe plumbing is concerned. The data is written to a buffer and retrieved as a (bigger) blob as and when (as already well described) the context completes allowing the event loop to continue.
The first data event seen below pushes not 1 but 5 chunks written to stdout by the child process whilst we were blocking.. thus nothing is lost.
sample:
let t = () => (new Date()).toTimeString().split(' ')[0]
let p = new Promise(function (resolve, reject) {
console.log(`[${t()}|info] spawning`);
let cp = spawn('bash', ['-c', 'for x in `seq 1 1 10`; do printf "$x\n"; sleep 1; done']);
let resolved = false;
if (cp === undefined)
reject();
cp.on('error', (err) => {
console.log(`error: ${err}`);
reject(err);
});
cp.stdout.on('data', (data) => {
if (!resolved) {
console.log(`[${t()}|info] spawn succeeded`);
resolved = true;
resolve();
}
process.stdout.write(`[${t()}|data] ${data}`);
});
let ts = parseInt(Date.now() / 1000);
while (parseInt(Date.now() / 1000) - ts < 5) {
// waste some cycles in the current context
ts--; ts++;
}
console.log(`[${t()}|info] synchronous time wasted`);
});
Promise.resolve(p);
output:
[18:54:18|info] spawning
[18:54:23|info] synchronous time wasted
[18:54:23|info] spawn succeeded
[18:54:23|data] 1
2
3
4
5
[18:54:23|data] 6
[18:54:24|data] 7
[18:54:25|data] 8
[18:54:26|data] 9
[18:54:27|data] 10

It is not a race condition. Node.js is single threaded and handles events on a first come first serve basis. New events are put at the end of the event loop. Node will execute your code in a synchronous manner, part of which will involve setting up event emitters. When these event emitters emit events, they will be put to the end of the queue, and will not be handled until Node finishes executing whatever piece of code its currently working on, which happens to be the same code that registers the listener. Therefore, the listener will always be registered before the event is handled.

Related

Wait for process.kill to terminate the process

My application terminates a process (exe file) then attempts to replace it with an updated version, I'm using process.kill with the pid of the process, I keep getting an error when trying to replace it with a newer version because the exe file is still in use and cannot be deleted, I have "resolved" this by waiting for 500ms but I wouldn't call that a good solution, I was expecting the method to be synchronous or at least have a sync counterpart just like the rest of fs methods.
Are there any other ways to do it in node.js?
The docs says the following :
Even though the name of this function is process.kill(), it is really just a signal sender, like the kill system call. The signal sent may do something other than kill the target process.
To me, it implies that process.kill will not indicate that the process is killed, but just that the signal has been sent (when it returns).
But there's another interesting line :
This method will throw an error if the target pid does not exist. As a special case, a signal of 0 can be used to test for the existence of a process. Windows platforms will throw an error if the pid is used to kill a process group.
So you can come up with something like that :
const killProcess = ({pid, signal = 'SIGTERM', timeout} = {}) => new Promise((resolve, reject) => {
process.kill(pid, signal);
let count = 0;
setInterval(() => {
try {
process.kill(pid, 0);
} catch (e) {
// the process does not exists anymore
resolve();
}
if ((count += 100) > timeout) {
reject(new Error("Timeout process kill"))
}
}, 100)
})

child_process.on('close') is sometimes painfully slow

I have a electron application that opens a external program (in my case Office), and has to wait for the program to be closed.
the code I wrote works great but sometimes the child_process.on('close') event is fired 10 or 20 seconds after the program has closed. The code is:
const cp = require("child_process");
child = cp.spawn(path/to/Office.exe + ' "' + path/to/myFile.pptx + '"', {shell: true});
child.on('close', function (code) {
//do something
});
Most of the time it reacts after 1 or 2 seconds which is fine, but sometimes it takes up to 20 seconds until I receive the close event. The program closes fast (according to the task manager), but node seems to wait for something.
I also tried child.on('exit'), calling the program with cp.exec()and using the options.stdio: ignore for spawn, as I thought maybe node is waiting for some stream from the child. But that made no difference.
Does anybody know a safe way to speed that process up?
I have tried your code and the close event triggers with a 0.5-2s delay, bearable i would say.
However, the 20s delay did not occur, but if this problem still persists on your end, you can try the approach below, which consists in checking the spawn pid.
const pidExists = (pid) => {
let pidOk = true;
try {
process.kill(pid, 0);
} catch (e) {
pidOk = false;
}
return pidOk;
};
const cp = require("child_process");
// I added the detach option because we won't need that process anymore since we're following the PID.
let child = cp.spawn(path/to/Office.exe + ' "' + path/to/myFile.pptx + '"', {shell: true, detach: true});
let officePID = child.pid; // this is the spawn pid
setInterval(()=>{
if( pidExists(officePID)){
console.log('file is still open', new Date().getTime());
}else{
console.log('file was closed', new Date().getTime());
process.exit(0);
}
}, 500);
This is a better approach since you said that the task manager shows you that the program was closed.

Electron kill child_process.exec

I have an electron app that uses child_process.exec to run long running tasks.
I am struggling to manage when the user exits the app during those tasks.
If they exit my app or hit close the child processes continue to run until they finish however the electron app window has already closed and exited.
Is there a way to notify the user that there are process still running and when they have finished then close the app window?
All I have in my main.js is the standard code:
// Quit when all windows are closed.
app.on('window-all-closed', function() {
// On OS X it is common for applications and their menu bar
// to stay active until the user quits explicitly with Cmd + Q
if (process.platform != 'darwin') {
app.quit();
}
});
Should I be adding a check somewhere?
Thanks for your help
EDITED
I cannot seem to get the PID of the child_process until it has finished. This is my child_process code
var loader = child_process.exec(cmd, function(error, stdout, stderr) {
console.log(loader.pid)
if (error) {
console.log(error.message);
}
console.log('Loaded: ', value);
});
Should I be trying to get it in a different way?
So after everyones great comments I was able to update my code with a number of additions to get it to work, so am posting my updates for everyone else.
1) Change from child_process.exec to child_process.spawn
var loader = child_process.spawn('program', options, { detached: true })
2) Use the Electron ipcRenderer to communicate from my module to the main.js script. This allows me to send the PIDs to main.js
ipcRenderer.send('pid-message', loader.pid);
ipcMain.on('pid-message', function(event, arg) {
console.log('Main:', arg);
pids.push(arg);
});
3) Add those PIDs to array
4) In my main.js I added the following code to kill any PIDs that exist in the array before exiting the app.
// App close handler
app.on('before-quit', function() {
pids.forEach(function(pid) {
// A simple pid lookup
ps.kill( pid, function( err ) {
if (err) {
throw new Error( err );
}
else {
console.log( 'Process %s has been killed!', pid );
}
});
});
});
Thanks for everyones help.
ChildProcess emits an exit event when the process has finished - if you keep track of the current processes in an array, and have them remove themselves after the exit event fires, you should be able to just foreach over the remaining ones running ChildProcess.kill() when you exit your app.
This may not be 100% working code/not the best way of doing things, as I'm not in a position to test it right now, but it should be enough to set you down the right path.
var processes = [];
// Adding a process
var newProcess = child_process.exec("mycommand");
processes.push(newProcess);
newProcess.on("exit", function () {
processes.splice(processes.indexOf(newProcess), 1);
});
// App close handler
app.on('window-all-closed', function() {
if (process.platform != 'darwin') {
processes.forEach(function(proc) {
proc.kill();
});
app.quit();
}
});
EDIT: As shreik mentioned in a comment, you could also just store the PIDs in the array instead of the ChildProcess objects, then use process.kill(pid) to kill them. Might be a little more efficient!
Another solution. If you want to keep using exec()
In order to kill the child process running by exec() take a look to the module ps-tree. They exaplain what is happening.
in UNIX, a process may terminate by using the exit call, and it's
parent process may wait for that event by using the wait system call.
the wait system call returns the process identifier of a terminated
child, so that the parent tell which of the possibly many children has
terminated. If the parent terminates, however, all it's children have
assigned as their new parent the init process. Thus, the children
still have a parent to collect their status and execution statistics.
(from "operating system concepts")
SOLUTION: use ps-tree to get all processes that a child_process may have started, so that they
exec() actually works like this:
function exec (cmd, cb) {
spawn('sh', ['-c', cmd]);
...
}
So check the example and adapt it to your needs
var cp = require('child_process'),
psTree = require('ps-tree');
var child = cp.exec("node -e 'while (true);'", function () { /*...*/ });
psTree(child.pid, function (err, children) {
cp.spawn('kill', ['-9'].concat(children.map(function (p) { return p.PID })));
});

Prevent sending data to stdin if spawn fails

In my Node.js (v0.10.9) code I'm trying to detect 2 cases:
an external tool (dot) is installed - in that case I want to send some data to stdin of created process
the external tool is not installed - in that case I want to display warning and I don't want to send anything to process' stdin
My problem is that I don't know how to send data to child's stdin if and only if the process was spawned successfully (i.e. stdin is ready for writing).
Following code works fine if dot is installed, but otherwise it tries to send data to the child although the child wasn't spawned.
var childProcess = require('child_process');
var child = childProcess.spawn('dot');
child.on('error', function (err) {
console.error('Failed to start child process: ' + err.message);
});
child.stdin.on('error', function(err) {
console.error('Working with child.stdin failed: ' + err.message);
});
// I want to execute following lines only if child process was spawned correctly
child.stdin.write('data');
child.stdin.end();
I'd need something like this
child.on('successful_spawn', function () {
child.stdin.write('data');
child.stdin.end();
});
From the node.js docs: http://nodejs.org/api/child_process.html#child_process_child_process_spawn_command_args_options
Example of checking for failed exec:
var spawn = require('child_process').spawn,
child = spawn('bad_command');
child.stderr.setEncoding('utf8');
child.stderr.on('data', function (data) {
if (/^execvp\(\)/.test(data)) {
console.log('Failed to start child process.');
}
});
Have a look at core-worker:
https://www.npmjs.com/package/core-worker
This package makes it a lot easier to handle processes.
I think what you want to do is something like that (from the docs):
import { process } from "core-worker";
const simpleChat = process("node chat.js", "Chat ready");
setTimeout(() => simpleChat.kill(), 360000); // wait an hour and close the chat
simpleChat.ready(500)
.then(console.log.bind(console, "You are now able to send messages."))
.then(::simpleChat.death)
.then(console.log.bind(console, "Chat closed"))
.catch(() => /* handle err */);
So if the process is not started correctly, none of the .then statements are executed which is exactly what you want to do, right?

Force Node.js to flush writes to child processes

I spawn a child process like this:
var child = require('child_process');
var proc = child.spawn('python', ['my_script.py', '-p', 'example']);
I also set some data handling:
proc.stdin.setEncoding('utf8');
proc.stdout.setEncoding('utf8');
proc.stderr.setEncoding('utf8');
proc.stdout.on('data', function (data) {
console.log('out: ' + data);
});
proc.stderr.on('data', function (data) {
console.log('err: ' + data);
});
proc.on('close', function (code) {
console.log('subprocess exited with status ' + code);
proc.stdin.end();
});
My Python script reads lines from stdin and for each line does some operations and prints to stdout. It works fine in the shell (I write a line and I get the output immediately) but when I do this in Node:
for (var i = 0; i < 10; i++) {
proc.stdin.write('THIS IS A TEST\n');
}
I get nothing.
I got to fix it calling proc.stdin.end() but that also terminates the child process (which I want to stay in background, streaming data).
I also triggered a flush filling the buffer with lots of writes, but that's not really an option.
Is there any way to manually flush the stream?
You are not flushing the output from Python after print statement. I had similar problem and #Alfe answered my question. Take a look at this:
Stream child process output in flowing mode

Resources