I would like to know if it is possible to wait for all child process created using the spawn function to finish before continuing execution.
I have a code looking like this:
const spawn = window.require('child_process').spawn;
let processes = [];
let thing = [];
// paths.length = 2
paths.forEach((path) => {
const pythonProcess = spawn("public/savefile.py", ['-d', '-j', '-p', path, tempfile]);
pythonProcess.on('exit', () => {
fs.readFile(tempfile, 'utf8', (err, data) => {
thing.push(...)
});
});
processes.push(pythonProcess);
});
console.log(processes) // Here we have 2 child processes
console.log(thing) // empty array.. the python processes didnt finish yet
return thing // of course it doesn't work. I want to wait for all the processes to have finished their callbacks to continue
As you can guess, I would like to know how I could get all the python scripts running at the same time, and wait for all of them to finish to continue my js code.
I'm running node 10.15.3
Thank you
ForEach to push Promise into an array of Promise and Promise.all()
Have you tried spawnSync ?
Is generally identical to spawn with the exception that the function
will not return until the child process has fully closed.
import { spawnSync } from "child_process";
spawnSync('ls', ['-la']);
Related
I would like to start a node-script called myScript.js, that starts a child process npm start, store the stdout of npm start into a global variable let myVar, and make sure that if the main program myScript.js is exited for any reason, the child process is killed as well. Nothing from the child's stdout should appear in the terminal window after ctr-c or similar.
My current solution does not kill on close:
const childProcess = require('child_process');
let myVar = ''
const child = childProcess.spawn('npm', ['start'], {
detached: false
});
process.on('exit', function () {
child.stdin.pause();
child.kill();
});
child.stdout.on('data', (data) => {
myVar = `${data}`
});
Can this be accomplished?
Small change, but I think that might look something like this:
const childProcess = require('child_process')
const child = childProcess.spawn('npm', ['start'], {shell:true});
var myVar = ''; child.stdout.setEncoding('utf8');
child.stdout.on('data', function(data) {
myVar = data.toString();
});
child.on('close', function(exitcode) {
// on the close of the child process, use standard output or maybe call a function
});
process.on('exit', function() {
// I don't think pausing std.in is strictly necessary
child.kill()
})
Further reading
node's child_process documentation, including event names
geeksforgeeks article on child_process.spawn
This StackOverflow thread on getting and using the output of a child_process' pipe
This StackOverflow thread on the difference between the close and exit events
I need to pass command-line arguments or params or execargv which should come in process params while loading environment for a child process which will be launched by a bull for process a job.
is it possible? if yes is there any way to do it?
I can identify the child process is launched for bull usings args[1] which contains /bull/lib/process
but I want to pass custom param to node process.
when a worker script runs, it reads the environment and keeps it until you shut it down.
If you need variable parameters to the function the worker should use, then it is best you send them in your queue.
queue.js
queue.add("foo", {params:"parameters you need", payload:{ foo: "bar" }});
worker.js
const worker = new Worker("foo",
async (job) => {
await your_function(job.data.params, job.data.payload);
}
);
const your_function = async (params, payload) => {
require("fs").writeFileSync("runner.json", JSON.stringify(payload), "utf8");
await require("child_process").fork("runner.js", params.split(" "));
};
runner.js
console.log(process.argv);
const fs = require("fs");
fs.readFile("runner.json", "utf8", function (err, data) {console.log("data: ", JSON.parse(data));});
I want to capture the data written to stderr of child process in parent.
const cluster = require('cluster');
if (cluster.isMaster) {
const child = cluster.fork();
cluster.workers[child.id].process.stderr.on('data',(data)=>{console.log(data});
}
else {
console.error("Child data");
}
I tried to add a callback to the child process stderr in master. But stderr is null in the parent.
How can I capture "Child data"(stderr of child process) in parent?
The requirement is to use cluster node module and capture console.error. I can not use process.send to send data to parent.
I prefer to do this with a Promise so I promisify the exec command like this:
const util = require("util");
const exec = util.promisify(require("child_process").exec);
let runCommand = async (cmd) => {
const result = await exec(cmd)
if (result.stdout) console.log("StdOut:",result.stdout)
if (result.stderr) console.log("StdErr:",result.stderr)
return result
}
runCommand("some command")
.then( result => console.log("Finished successfully),
reason => console.log("Failed with",reason.code"))
If an error is returned (non-zero) then the promise will not resolve and you can handle the rejection also.
I have a Node script that calls an external program (PluginManager.exe) this way:
const util = require('util');
const execFile = util.promisify(require('child_process').execFile);
const process = execFile('PluginManager.exe', ['/install']);
process
.then(({stdout, stderr}) => console.log('done', stdout, stderr))
.catch(e => console.log(e));
PluginManager.exe takes 8 seconds to execute. My problem is that the Node script keeps running for another 10 more seconds after the child process has exited. I know when PluginManager.exe finishes because I can see it disappear from the Windows Task Manager Process List.
What keeps the Node process running for so long and what can I do to make sure it exits as soon as the child process exits?
Maybe it's waiting on input and timing out after 10s?
Try closing stdin with .end() as mentioned in https://nodejs.org/api/child_process.html#child_process_subprocess_stdin
(in this usage, you'll need the original return value of execFile, so don't promisify, as per https://stackoverflow.com/a/30883005/1105015)
e.g.
const util = require('util');
const execFile = require('child_process').execFile;
const process = execFile(
'PluginManager.exe', ['/install'], (e, stdout, stderr) => {
if (e) {
console.log(e);
} else {
console.log('done', stdout, stderr));
}});
process.stdin.end();
Have you tried by adding the killSignal option set to something a bit more aggresive?
const process = execFile('PluginManager.exe', ['/install'], {killSignal: 'SIGKILL'});
I am spawning a child process in node where for the opts I'm using {stdio:'inherit'}, I need to do this because the child process needs to accept some input from the user.
I need to also get this output though, because the output of the child is actually nothing because it is using the parent's. I thought about doing a quick process.stdout.on('data', ...) attach and detach after the child process, but I prefer not to do that. I assume that is some node stream solution? Like something that would get intercept the input first, and then pass it along to my parent process?
TLDR Need output of child process while still doing stdio:'inherit' in node. Perhaps something that pipes output of child process to a Buffer/string?
Thank you and please if answering provide working code example
Figured it out!
const child_process = require('child_process');
const child = child_process.spawn('python3', ['./eater.py'], {
stdio: ['inherit', 'pipe', 'pipe'],
});
const output = [];
child.stdout.on('data', d => {
console.log(d.toString());
output.push(d.toString());
});
child.stdout.on('end', () => {
console.log('Finished');
console.log({ output });
});
and
if the python was something like:
print("Some prompt")
auth_code = input('Provide some code to us\n')