I want to execute a command like "doSomething ./myfiles/*.csv" with spawn in node.js. I want to use spawn instead of exec, because it is some kind of watch process and I need the stdout output.
I tried this
var spawn = require('child_process').spawn;
spawn("doSomething", ["./myfiles/*.csv"]);
But then the wildcard *.csv will not interpreted.
Is it not possible to use wildcards when using spawn()? Are there other possibilities to solve this problem?
Thanks
Torben
The * is being expanded by the shell, and for child_process.spawn the arguments are coming through as strings so will never get properly expanded. It's a limitation of spawn. You could try child_process.exec instead, it will allow the shell to expand any wildcards properly:
var exec = require("child_process").exec;
var child = exec("doSomething ./myfiles/*.csv",function (err,stdout,stderr) {
// Handle result
});
If you really need to use spawn for some reason perhaps you could consider expanding the wildcard file pattern yourself in Node with a lib like node-glob before creating the child process?
Update
In the Joyent Node core code we can observe an approach for invoking an arbitrary command in a shell via spawn while retaining full shell wildcard expansion:
https://github.com/joyent/node/blob/937e2e351b2450cf1e9c4d8b3e1a4e2a2def58bb/lib/child_process.js#L589
And here's some pseudo code:
var child;
var cmd = "doSomething ./myfiles/*.csv";
if ('win32' === process.platform) {
child = spawn('cmd.exe', ['/s', '/c', '"' + cmd + '"'],{windowsVerbatimArguments:true} );
} else {
child = spawn('/bin/sh', ['-c', cmd]);
}
Here's the simplest solution:
spawn("doSomething", ["./myfiles/*.csv"], { shell: true });
As #JamieBirch suggested in his comment, the key is telling spawn() to use the shell ({ shell: true }, see the docs), so the wildcard is properly resolved.
What OS are you using? In Unix-family OSs (e.g. Linux, MacOS), programs expect the shell process to expand wildcard filename arguments and pass the expansion in argv[]. In Windows OSs, programs usually expect to have to expand wildcards themselves (though only if they're Windows-native programs; ported Unix-family programs may at most try to run the arguments through a compatibility layer).
Your syntax looks like it's for a Unix-family system. If so, then when you call spawn() you're bypassing shell expansion, and your child process is going to treat dots and asterisks in arguments literally. Try using sh child_process in place of child_process and see if you get better results.
Related
If we fork a child_process in Node, how can we pass node parameters to the child_process?
https://nodejs.org/api/child_process.html
Specifically I would like to spawn ~20 processes, and would like to limit the memory usage of each by using --v8-options, but I can't find any examples of doing this - is this possible or do the child processes assume the same node parameters as the parent?
the parent would be:
node foo.js
and the children would be
node --some-flag=bar baz.js
...
I am looking to pass node options using
child_process.fork()
but if it's only possible with
spawn()
or
exec()
then I guess I will take what I can get.
As a simple example, the following will not run Node.js with the --harmony flag
var cp = require('child_process');
var args = ['--harmony'];
var n = cp.fork(filePath, args , Object.create(process.env));
You'll need to set the execArgv option to fork.
If you don't, you'll get the same option as the node process you're 'forking' (it actually is just a spawn, not a POSIX-fork).
So you could do something like this:
var n = cp.fork(modname, {execArgv: ['--harmony']});
If you want to pass on the node-options from the parent:
var n = cp.fork(modname, {execArgv: process.execArgv.concat(['--harmony'])}
Warning: child_process has a safeguard against the -e switch that you are circumventing with that! So don't do this from the command line with an -e or -p.
You will be creating a new process with a script that is the same as that from the parent – a fork bomb.
If you still want to be able to pass on options to fork via the environment, you could do something like this:
var cp = require('child_process');
var opts = Object.create(process.env);
opts.execArgv = ['--harmony'];
var n = cp.fork(filePath, opts);
Another option may be to alter process.execArgv (like process.execArgv.push('--harmony')) but I am pretty sure that is a bad idea and might result in strange behaviour elswhere.
I'm adding arguments to child_process.exec command using string concatenation and they are ignored
var exec = require( "child_process" ).exec;
var cmd = exec( "grunt build --project="+application, {
cwd: application
},
function( error, stdout, stderr ){});
cmd.stdout.pipe( process.stdout );
cmd.stderr.pipe( process.stderr );
Why is string concatenation a problem and how to avoid it?
Your code is vulnerable to command injection. It depends where's application coming from and you need to make sure it's not customizable by user.
Malicious code in your example would be
var application = '; rm -rf .'
but it wouldn't work since you're also trying to change the current directory via pwd.
The general recommendation is to be careful with child_process.exec and use child_process.execFile or child_process.spawn instead.
Check your grunt build task to see if there is anything wrong. There is nothing wrong in your code with string concatenation in child_process.exec
Is it possible to use the array version of perls system command (i.e. the version that takes the first element as the command and the rest of the array as the cmd arguments) while also spawning a new process with Linux so that the system command returns immediately e.g. to run a command like:
mycmd arg1 arg2 &
I've tried using system( ('mycmd', 'arg1', 'arg2', '&') ) but it just interprets the ampersand literally as a third argument.
I know I can just pass the whole command as a scalar to system but I'm specifically wondering if it's possible to use the array version because the parameters for this command will originate from user options in a CGI script.
The & part of the shell command tells the shell to run the process in the background, so bypassing the shell by using the multi-arg form of system makes no sense.
Solution 1: Quote using String::ShellQuote.
use String:ShellQuote qw( shell_quote );
system(shell_quote('mycmd', 'arg1', 'arg2').' &');
Solution 2: Quote using shell interpolation.
system('sh', '-c', '"$#" &', 'sh', 'mycmd', 'arg1', 'arg2');
Solution 3: Launch the program in the background yourself.
use IPC::Open3 qw( open3 );
{
open(local *CHILD_IN, '<', '/dev/null') or die $!;
local $SIG{CHLD} = 'IGNORE';
open3(
'<&CHILD_IN', '>&STDOUT', '>&STDERR',
'mycmd', 'arg1', 'arg2',
);
}
Since you have no interest in the fate of the executed program you can use fork/exec. And you're on Linux which allows using $SIG{CHLD} = 'IGNORE' to avoid waiting on the child process.
sub background {
local $SIG{CHLD} = 'IGNORE';
# fork and then exec if we are the child
exec(#_) or die($!) unless fork;
}
background( 'mycmd', 'arg1', 'arg2' );
I am trying to open a file through the command line by using node. I am using child_process.spawn, here is the code
var
process = require('child_process'),
cmd = process.spawn('cmd', ['start','tmp.txt'], {cwd: 'C:\\Users\\testuser\\Node_dev'});
I am expecting the file tmp.txt located in the Node_dev folder to be opened but I am getting the error -
dir error { [Error: spawn ENOENT] code: 'ENONENT', errno: 'ENOENT', syscall: 'spawn'
What am I missing?
Also, what's the difference between child_process.spawn vs child_process.exec for this?
I don't have an explanation for your ENOENT error [update: turns out this was a red herring], but one problem with your command is that you launch cmd.exe without /c, meaning that the shell spawned will (a) stay open and (b) actually ignore the specified command.
As for how the child_process module's methods differ:
.execFile and .exec take a callback that reports a single, buffered result, whereas .spawn provides chunked output via events.
Only .exec passes the command to the platform's default shell, which makes it more convenient, but less efficient.
In your case, you don't care about output returned from the spawned command, and since you need to involve the shell anyway,.exec is the best choice.
On Windows, use of start comes with its own pitfalls:
It is not an executable, but a shell builtin (to put it in Unix terms) - thus, it must be invoked via cmd.exe.
If its 1st argument is a [double-quoted] value with embedded spaces, it is interpreted as a window title rather than a filename argument; thus, to robustly pass a filename argument, you must pass an empty window title as the 1st argument.
Here are invocations that should work on Windows - note that both launch the child process asynchronously and ignore any output from it - all they do is to tell cmd.exe to open the specified file as if a user had opened it in Explorer:
using .exec:
var cpm = require('child_process');
// With .exec, specify the entire shell command as the 1st argument - it is implicitly
// passed to cmd.exe.
// '""' as the 1st argument to `start` is an empty window title that ensures that any
// filename argument with embedded spaces isn't mistaken for a window title.
cpm.exec('start "" "tmp.txt"', {cwd: 'C:\\Users\\testuser\\Node_dev'});
using .execFile or .spawn:
// With .spawn or .execFile, specify `cmd` as the 1st argument, and the shell command
// tokens as an array passed as the 2nd argument.
// Note the /c, which ensures that cmd exits after having executed the specified
// command.
// '""' as the 1st argument to `start` is an empty window title that ensures that any
// filename argument with embedded spaces isn't mistaken for a window title.
cpm.spawn('cmd', [ '/c', 'start', '""', 'tmp.txt' ], {cwd: 'C:\\Users\\testuser\\Node_dev'});
There is a cross-platform module on npm called open that opens files using the OS's default handler that you could use as a guide or just use it as-is.
To do it manually with spawn on Windows though, you'd do something like:
var args = ['/s', '/c', 'start', '', 'tmp.txt'],
opts = { cwd: 'C:\\Users\\testuser\\Node_dev' },
child = child_process.spawn('cmd.exe', args, opts);
child_process.exec() is basically a higher-level wrapper for child_process.spawn() which buffers output and then passes the buffered stdout and stderr output to a callback.
I am using the same, here is my command to run and executable with arguments:
var exec = require('child_process').spawn;
let scanProcess = exec('cmd', [ '/c', 'start', '""', 'feature.exe --version' ], {cwd: 'command_folder'});
I am running this on Azure DevOps in a container running Windows OS.
The above command just doesn't come back, it hangs. Any pointers.
I want to execute a command like "doSomething ./myfiles/*.csv" with spawn in node.js. I want to use spawn instead of exec, because it is some kind of watch process and I need the stdout output.
I tried this
var spawn = require('child_process').spawn;
spawn("doSomething", ["./myfiles/*.csv"]);
But then the wildcard *.csv will not interpreted.
Is it not possible to use wildcards when using spawn()? Are there other possibilities to solve this problem?
Thanks
Torben
The * is being expanded by the shell, and for child_process.spawn the arguments are coming through as strings so will never get properly expanded. It's a limitation of spawn. You could try child_process.exec instead, it will allow the shell to expand any wildcards properly:
var exec = require("child_process").exec;
var child = exec("doSomething ./myfiles/*.csv",function (err,stdout,stderr) {
// Handle result
});
If you really need to use spawn for some reason perhaps you could consider expanding the wildcard file pattern yourself in Node with a lib like node-glob before creating the child process?
Update
In the Joyent Node core code we can observe an approach for invoking an arbitrary command in a shell via spawn while retaining full shell wildcard expansion:
https://github.com/joyent/node/blob/937e2e351b2450cf1e9c4d8b3e1a4e2a2def58bb/lib/child_process.js#L589
And here's some pseudo code:
var child;
var cmd = "doSomething ./myfiles/*.csv";
if ('win32' === process.platform) {
child = spawn('cmd.exe', ['/s', '/c', '"' + cmd + '"'],{windowsVerbatimArguments:true} );
} else {
child = spawn('/bin/sh', ['-c', cmd]);
}
Here's the simplest solution:
spawn("doSomething", ["./myfiles/*.csv"], { shell: true });
As #JamieBirch suggested in his comment, the key is telling spawn() to use the shell ({ shell: true }, see the docs), so the wildcard is properly resolved.
What OS are you using? In Unix-family OSs (e.g. Linux, MacOS), programs expect the shell process to expand wildcard filename arguments and pass the expansion in argv[]. In Windows OSs, programs usually expect to have to expand wildcards themselves (though only if they're Windows-native programs; ported Unix-family programs may at most try to run the arguments through a compatibility layer).
Your syntax looks like it's for a Unix-family system. If so, then when you call spawn() you're bypassing shell expansion, and your child process is going to treat dots and asterisks in arguments literally. Try using sh child_process in place of child_process and see if you get better results.