I'm adding arguments to child_process.exec command using string concatenation and they are ignored
var exec = require( "child_process" ).exec;
var cmd = exec( "grunt build --project="+application, {
cwd: application
},
function( error, stdout, stderr ){});
cmd.stdout.pipe( process.stdout );
cmd.stderr.pipe( process.stderr );
Why is string concatenation a problem and how to avoid it?
Your code is vulnerable to command injection. It depends where's application coming from and you need to make sure it's not customizable by user.
Malicious code in your example would be
var application = '; rm -rf .'
but it wouldn't work since you're also trying to change the current directory via pwd.
The general recommendation is to be careful with child_process.exec and use child_process.execFile or child_process.spawn instead.
Check your grunt build task to see if there is anything wrong. There is nothing wrong in your code with string concatenation in child_process.exec
Related
I am trying to execute an external bash script from nodejs.
The script when executed from terminal, prompts for input, such as a password.
When executing from nodejs, i can not get it to pause to read the input.
Can not find any good info online on this.
Is this not possible?
var shelljs = require( 'shelljs' )
var returns = shelljs.exec("passwd", {shell: '/bin/bash'}, {stdio:'inherit'});
passwd: conversation failure
shelljs.exec("exec </dev/tty\n passwd", ... )
seems to work. found by accident.
Not sure if this is the best way though.
Not very solid i think.
I have been using shelljs
On my super fast system I execute this:
var shell = require('shelljs')
const exec = require('child_process').exec
console.time('shell mktemp -d')
shell.exec('mktemp -d', {silent: true})
console.timeEnd('shell mktemp -d')
console.time('child exec mktemp -d')
exec('mktemp', ['-d'], function(error, stdout, stderr) {
if (error) {
console.error('stderr', stderr)
throw error
}
console.log('exec stdout', stdout)
console.timeEnd('child exec mktemp -d')
})
Its giving the following execution times:
shell mktemp -d: 208.126ms
exec stdout /tmp/tmp.w22tyS5Uyu
child exec mktemp -d: 48.812ms
Why is shelljs 4 times slower? Any thoughts?
Your code example compares async child_process.exec() with sync shell.exec(), which isn't entirely a fair comparison. I think you'll find shell.exec(..., { async: true }) performs a bit better: this is because sync shell.exec() does extra work to provide real-time stdio while still capturing stdout/stderr/return code as part of its return value; async shell.exec() can provide the same feature mostly for free.
Even with { silent: true }, the extra work is still necessary. shell.exec() is built on top of child_process.execSync(), which only returns stdout. We need to perform the same extra work in order to return return code and stderr.
Have a look to how shelljs is implemented:
It fully relies on node.js fs library. This library is cross platform and written in C++ but not as performant as C language. More generally, you can't have in JS the perfs you get in C...
Another thing, abstraction layers:
you're using exec(Command) where Command is a C tailored (Linux C here I think). The machine creates a thread and executes a command in it.
When using shell.js, there are many mechanisms to ensure cross plateform and keep the abstraction of your command as a function and keep the result as a variable. See the code of exec in shell.js:
https://github.com/shelljs/shelljs/blob/master/src/exec.js
It is not really doing the same thing as your line of code.
Hope that helps!
I want to execute a command like "doSomething ./myfiles/*.csv" with spawn in node.js. I want to use spawn instead of exec, because it is some kind of watch process and I need the stdout output.
I tried this
var spawn = require('child_process').spawn;
spawn("doSomething", ["./myfiles/*.csv"]);
But then the wildcard *.csv will not interpreted.
Is it not possible to use wildcards when using spawn()? Are there other possibilities to solve this problem?
Thanks
Torben
The * is being expanded by the shell, and for child_process.spawn the arguments are coming through as strings so will never get properly expanded. It's a limitation of spawn. You could try child_process.exec instead, it will allow the shell to expand any wildcards properly:
var exec = require("child_process").exec;
var child = exec("doSomething ./myfiles/*.csv",function (err,stdout,stderr) {
// Handle result
});
If you really need to use spawn for some reason perhaps you could consider expanding the wildcard file pattern yourself in Node with a lib like node-glob before creating the child process?
Update
In the Joyent Node core code we can observe an approach for invoking an arbitrary command in a shell via spawn while retaining full shell wildcard expansion:
https://github.com/joyent/node/blob/937e2e351b2450cf1e9c4d8b3e1a4e2a2def58bb/lib/child_process.js#L589
And here's some pseudo code:
var child;
var cmd = "doSomething ./myfiles/*.csv";
if ('win32' === process.platform) {
child = spawn('cmd.exe', ['/s', '/c', '"' + cmd + '"'],{windowsVerbatimArguments:true} );
} else {
child = spawn('/bin/sh', ['-c', cmd]);
}
Here's the simplest solution:
spawn("doSomething", ["./myfiles/*.csv"], { shell: true });
As #JamieBirch suggested in his comment, the key is telling spawn() to use the shell ({ shell: true }, see the docs), so the wildcard is properly resolved.
What OS are you using? In Unix-family OSs (e.g. Linux, MacOS), programs expect the shell process to expand wildcard filename arguments and pass the expansion in argv[]. In Windows OSs, programs usually expect to have to expand wildcards themselves (though only if they're Windows-native programs; ported Unix-family programs may at most try to run the arguments through a compatibility layer).
Your syntax looks like it's for a Unix-family system. If so, then when you call spawn() you're bypassing shell expansion, and your child process is going to treat dots and asterisks in arguments literally. Try using sh child_process in place of child_process and see if you get better results.
Currently I'm failing to fetch the rsync output when I'm calling nodejs child_process.exec with a callback-function like in this snippet:
var sys = require('sys'),
exec = require('child_process').exec;
cmd = 'rsync -rpz test/test-files/one.txt jloos#test.mygnia.de:~/remote-test/a/b/'
exec(cmd, function(error, stdio, stderr) {
sys.print('s: ' + stdio + '\n');
sys.print('e: ' + stderr + '\n');
});
I think this is caused by the specific behavior of rsync. rsync communicates with it's counterpart via terminal. So how can I fetch the messages from rsync, if even possible?
When I use cmd = 'ls -la' I get the expected output.
Thanks
Often stdout is buffered when the program isn't running in a virtual terminal.
Many languages have a pty module which will trick the program into behaving as though it is running in a terminal.
This provides that functionality for NodeJs;
https://github.com/chjj/pty.js
Keep in mind that rsync may be writing lots of special characters or using something like ncurses to provide the updating status messages, which may make it more difficult to work with the output.
I want to execute a command like "doSomething ./myfiles/*.csv" with spawn in node.js. I want to use spawn instead of exec, because it is some kind of watch process and I need the stdout output.
I tried this
var spawn = require('child_process').spawn;
spawn("doSomething", ["./myfiles/*.csv"]);
But then the wildcard *.csv will not interpreted.
Is it not possible to use wildcards when using spawn()? Are there other possibilities to solve this problem?
Thanks
Torben
The * is being expanded by the shell, and for child_process.spawn the arguments are coming through as strings so will never get properly expanded. It's a limitation of spawn. You could try child_process.exec instead, it will allow the shell to expand any wildcards properly:
var exec = require("child_process").exec;
var child = exec("doSomething ./myfiles/*.csv",function (err,stdout,stderr) {
// Handle result
});
If you really need to use spawn for some reason perhaps you could consider expanding the wildcard file pattern yourself in Node with a lib like node-glob before creating the child process?
Update
In the Joyent Node core code we can observe an approach for invoking an arbitrary command in a shell via spawn while retaining full shell wildcard expansion:
https://github.com/joyent/node/blob/937e2e351b2450cf1e9c4d8b3e1a4e2a2def58bb/lib/child_process.js#L589
And here's some pseudo code:
var child;
var cmd = "doSomething ./myfiles/*.csv";
if ('win32' === process.platform) {
child = spawn('cmd.exe', ['/s', '/c', '"' + cmd + '"'],{windowsVerbatimArguments:true} );
} else {
child = spawn('/bin/sh', ['-c', cmd]);
}
Here's the simplest solution:
spawn("doSomething", ["./myfiles/*.csv"], { shell: true });
As #JamieBirch suggested in his comment, the key is telling spawn() to use the shell ({ shell: true }, see the docs), so the wildcard is properly resolved.
What OS are you using? In Unix-family OSs (e.g. Linux, MacOS), programs expect the shell process to expand wildcard filename arguments and pass the expansion in argv[]. In Windows OSs, programs usually expect to have to expand wildcards themselves (though only if they're Windows-native programs; ported Unix-family programs may at most try to run the arguments through a compatibility layer).
Your syntax looks like it's for a Unix-family system. If so, then when you call spawn() you're bypassing shell expansion, and your child process is going to treat dots and asterisks in arguments literally. Try using sh child_process in place of child_process and see if you get better results.