This is my NodeJs snippet:
const {execSync} = require('child_process');
testoutput = execSync('make test');
Here I am getting stdout as output to testoutput variable from the execSync process if the make process is completely exited. But, if the make test process encounters an stderr error then I am getting an error in NodeJs.
How can I store the stderr message to an output variable testoutput if the make process does not exit and store stdout in variable testoutput if the process exits well with stdout output.
To catch errors with execSync(), you need a try/catch around your call as errors will throw an exception.
Related
I have to run a python script from my node project that converts a .csv file to a .txt file.
The command for running the python script on the terminal is
python3 csv_to_rdf.py "pathToFile" "date" > "nameOfNewFile.txt"
eg. python3 csv_to_rdf.py modifiedCSV.csv 08122022 > test.txt
I am using the child_process.spawn() method in nodeJS to run the script but I can't figure out the args.
When I use the following code snippet:
I get the following error message:
My question is how do I send in "> test.txt" in my spawn() method?
This is incredible, I am a semantic developer, and I had an extremely similar file (a python script that parsed a .csv and then converted it to .txt and then to .TriG) and I needed it run in my Node.js backend.
Short answer
A very simple solution I thought of after writing everything below is:
Python is better at writing to files than bash. Use python to write to your file and then do not pass a redirect into your spawn() and you'll be good. Escaped characters are difficult to handle in the child_process because you have to deal with JavaScript and Python both trying to handle escaped characters at the same time. If python doesn't have write access, then below may server you better.
Long answer
I moved away from child processes because they cause security issues. Any child_process with user input can be exploited. If you could move away from a child process and re-write your python file in JS/TypeScript, that would be best. If you are confident in the use of the spawn, then here is a solution:
Firstly, you cannot add a redirect to your spawn because the command is python3 and, as the error suggests, python3 takes the redirect as an unrecognized argument. It literally takes it as an argument to the python file, so sys.argv is the array you passed in the spawn. Python will take it in but without seeing your python code, I couldn't say how the extra arguments are being handled. Obviously an error is being thrown.
Remember the child_process.spawn() happens asynchronously, so the stdout and stderr can be extracted right after. Read child_process.spawn() docs. So remove the redirect in the spawns argument array and just run another child_process or use fn to write the output to a file when the stdout occurs:
const { spawn } = require('node:child_process');
const childPython = spawn('python3', ['modifiedCSV.csv', '08182022']);
childPython.stdout.on('data', (data) => {
console.log(`stdout: ${data}`);
});
Or you could use exec instead.
const { exec } = require('node:child_process');
const path = "modifiedCSV.csv";
const date = "08182022";
exec(`python3 ${path} ${date} > test.txt`, (error, stdout, stderr) => {
if (error) {
console.error(`exec error: ${error}`);
return;
}
console.log(`stdout: ${stdout}`);
console.error(`stderr: ${stderr}`);
})
The main issue is, if the path to the modified csv file is in anyway connected to a user's input, you have a security risk, because if python3 is run as sudo, which I believe in node, it is, a user can inject python code in-place of the file path and re-write and encrypt all the files that it can touch.
We have a VS Code extension built in node that runs like so:
Our extension is a node process, it calls child_process.spawn(...), with pipe option for the stdio. The child process' stdin and stdout are used to pass messages between the two processes.
Due to a bug in the child process binary, we see an issue where SIGSEGV error code is thrown by the binary. When we call into the process on the commandline and hit this error, we see the stack trace is dumped to stderr.
In the VS Code extension/node process, we see the handlers for the child process exiting are hit, however the handlers for the stderr output do not.
In other non-crashing scenarios, we see stderr output is correctly transmitted.
The implementation of spawning and handling the child process is reasonably complex, however it boils down to something like this:
child_process.ChildProcess childProcess;
startChildProcess() {
this.childProcess = child_process.spawn(binary, args, {
cwd: root,
env: {...process.env, ...environment},
// pipe stdin and stdout, and inherit stderr (unless parent know better)
stdio: ['pipe', 'pipe', 'pipe'],
});
// set up stdin and stdout pipes here...
this.childProcess.on('exit', (code, signal) => {
// do something interesting when the process exits
// THIS CODE BLOCK IS HIT
}
this.childProcess.stderr?.addListener('data', (errOutput) => {
const errOutputStr = errOutput.toString();
process.stderr.write(errOutputStr);
// THIS CODE BLOCK IS NOT HIT WHEN THE ABOVE IS HIT
})
}
As annotated in the example, the stderr output is not hit in the case of a SIGSEGV from child process.
What can I do to ensure the errors are output? VS Code, the extension process and the child process are all running on Mac OS
My node process gets some PDF file via HTTP Request, then uses the request's onData event to pass the incoming data on to a properly configured lpr, spawned via child_process.exec. I write to stdin using process.stdin.write(...), followed by process.stdin.end() when done. This allows me to print those files immediately.
Now I have a situation where I don't want the data to be piped to lpr, but to some bash script. The script uses cat to process its stdin.
myscript.sh < somefile.pdf works as expected, as does cat somefile.pdf | myscript.sh.
However, when I spawn /path/to/script.sh from node (by simply replacing lpr with the script path in the source), the process exits with
events.js:183
throw er; // Unhandled 'error' event
^
Error: write EPIPE
at WriteWrap.afterWrite [as oncomplete] (net.js:868:14)
Subsequently, the whole node process crashes, the error sneaking around all try...catch blocks. Logging at the beginning of the bash script shows, it does not even get started.
When I target anything that's not a shell script but some compiled executable, like cat, echo,... everything works just fine.
Adding epipebomb module would not change anything.
I also tried piping to process.exec("bash", ["-c cat | myscript.sh"]), with the same errors.
An example bash script, just to test for execution:
#!/usr/bin/env bash
date > logfile.txt
cat > /dev/null
EDIT:
I think I maybe need to signal to keep the stdin stream open somehow.
The process-spawning part of the script, leaving promisification and output processing away:
const process = require("child_process")
// inputObservable being an rxjs Observable
execstuff(inputObervable) {
const task = process.spawn("/path/to/script.sh");
inputObservable.subscribe(
chunk => task.stdin.write(chunk),
error => console.error(error),
finished => task.stdin.end()
);
}
There is an example at child_process.spawn how you can write the following lines ps ax | grep ssh as node.js script, maybe it will be helpful for you:
const { spawn } = require('child_process');
const ps = spawn('ps', ['ax']);
const grep = spawn('grep', ['ssh']);
ps.stdout.on('data', (data) => {
grep.stdin.write(data);
});
ps.stderr.on('data', (data) => {
console.log(`ps stderr: ${data}`);
});
The first impression is that you are doing the same stuff, the problem may be in the chunk data, maybe one of the chunks is null, and it is closing the stream, and you want to close it by running task.stdin.end().
The other thing you can try is to run the node.js script with the NODE_DEBUG=stream node script.js
Will log the node.js internals how the stream, behaves, also may be helpful for you.
What are the ways to flush a Linux command's stdout as a Node.js child process?
Ending the stdin stream (child.stdin.end) will work. As will unbuffering the command with stdbuf.
But I imagine there's a proper way to stream results from external commands.
How do we tell a command we're ready to consume while we are still providing data?
Example:
const { spawn } = require('child_process');
const child = spawn('uniq');
child.stdout.pipe(process.stdout);
child.stdin.write('a\nb\nb\nc\n', 'utf8');
// No output, child is still running.
(uniq is just an example here. It's the same with most Linux commands.)
I'm trying to wrap the lftp program in a node.js application, using child_process. The problem is that lftp doesn't write its output to stdout, so I cannot catch its output in node.js. Sample code:
var proc = require('child_process').spawn('lftp', ['-p', port, '-u', username + ',' + password, host]);
proc.stdout.on('data', function (data) {
console.log('stdout:', data.toString('utf-8'));
});
proc.on('exit', function (code) {
console.log('process exited with code ' + code);
});
proc.stdin.write('ls');
// proc.stdin.end();
If I uncomment the line that calls stdin.end() for the lftp child process, the output from the ls command appears in my terminal as it should. If I don't the process simply hangs and nothing gets outputted.
I've also tried using unbuffer, but it doesn't seem to allow me to write to lftp's stdin anymore. It outputs the usual "[Resolving host address...]" stuff, but not the output from the ls command.
My question is: what do I have to do to be able to interact with lftp using node.js' child_process?
Well, this was dumb. I forgot to write a newline after the ls command to stdin. It seems to work without the need for unbuffer.