I have been using shelljs
On my super fast system I execute this:
var shell = require('shelljs')
const exec = require('child_process').exec
console.time('shell mktemp -d')
shell.exec('mktemp -d', {silent: true})
console.timeEnd('shell mktemp -d')
console.time('child exec mktemp -d')
exec('mktemp', ['-d'], function(error, stdout, stderr) {
if (error) {
console.error('stderr', stderr)
throw error
}
console.log('exec stdout', stdout)
console.timeEnd('child exec mktemp -d')
})
Its giving the following execution times:
shell mktemp -d: 208.126ms
exec stdout /tmp/tmp.w22tyS5Uyu
child exec mktemp -d: 48.812ms
Why is shelljs 4 times slower? Any thoughts?
Your code example compares async child_process.exec() with sync shell.exec(), which isn't entirely a fair comparison. I think you'll find shell.exec(..., { async: true }) performs a bit better: this is because sync shell.exec() does extra work to provide real-time stdio while still capturing stdout/stderr/return code as part of its return value; async shell.exec() can provide the same feature mostly for free.
Even with { silent: true }, the extra work is still necessary. shell.exec() is built on top of child_process.execSync(), which only returns stdout. We need to perform the same extra work in order to return return code and stderr.
Have a look to how shelljs is implemented:
It fully relies on node.js fs library. This library is cross platform and written in C++ but not as performant as C language. More generally, you can't have in JS the perfs you get in C...
Another thing, abstraction layers:
you're using exec(Command) where Command is a C tailored (Linux C here I think). The machine creates a thread and executes a command in it.
When using shell.js, there are many mechanisms to ensure cross plateform and keep the abstraction of your command as a function and keep the result as a variable. See the code of exec in shell.js:
https://github.com/shelljs/shelljs/blob/master/src/exec.js
It is not really doing the same thing as your line of code.
Hope that helps!
Related
I am looking for a way to pass or output data from one script to another script so that the late script can execute itself with the output that came from the first one.
Basically, I have a ruby script with some instructions in it and I want to pass (or output...) the result of the ruby script to a node.js script.
I would like help ( and examples ... ) on how to realize this and/or recommendations for techniques or technologies I might have never heard of it that might do the trick
Thank you.
You can use child_process exec to execute a script and handle it's output.
Ruby Script
# example.rb
puts "hello world"
Node Script
// example.js
const exec = require('child_process').exec
exec('ruby example.rb', function(err, stdout, stderr) {
console.error(err)
console.error('stderr: ' + stderr)
console.log('stdout: ' + stdout) // logs "hello world"
});
I want to execute a command like "doSomething ./myfiles/*.csv" with spawn in node.js. I want to use spawn instead of exec, because it is some kind of watch process and I need the stdout output.
I tried this
var spawn = require('child_process').spawn;
spawn("doSomething", ["./myfiles/*.csv"]);
But then the wildcard *.csv will not interpreted.
Is it not possible to use wildcards when using spawn()? Are there other possibilities to solve this problem?
Thanks
Torben
The * is being expanded by the shell, and for child_process.spawn the arguments are coming through as strings so will never get properly expanded. It's a limitation of spawn. You could try child_process.exec instead, it will allow the shell to expand any wildcards properly:
var exec = require("child_process").exec;
var child = exec("doSomething ./myfiles/*.csv",function (err,stdout,stderr) {
// Handle result
});
If you really need to use spawn for some reason perhaps you could consider expanding the wildcard file pattern yourself in Node with a lib like node-glob before creating the child process?
Update
In the Joyent Node core code we can observe an approach for invoking an arbitrary command in a shell via spawn while retaining full shell wildcard expansion:
https://github.com/joyent/node/blob/937e2e351b2450cf1e9c4d8b3e1a4e2a2def58bb/lib/child_process.js#L589
And here's some pseudo code:
var child;
var cmd = "doSomething ./myfiles/*.csv";
if ('win32' === process.platform) {
child = spawn('cmd.exe', ['/s', '/c', '"' + cmd + '"'],{windowsVerbatimArguments:true} );
} else {
child = spawn('/bin/sh', ['-c', cmd]);
}
Here's the simplest solution:
spawn("doSomething", ["./myfiles/*.csv"], { shell: true });
As #JamieBirch suggested in his comment, the key is telling spawn() to use the shell ({ shell: true }, see the docs), so the wildcard is properly resolved.
What OS are you using? In Unix-family OSs (e.g. Linux, MacOS), programs expect the shell process to expand wildcard filename arguments and pass the expansion in argv[]. In Windows OSs, programs usually expect to have to expand wildcards themselves (though only if they're Windows-native programs; ported Unix-family programs may at most try to run the arguments through a compatibility layer).
Your syntax looks like it's for a Unix-family system. If so, then when you call spawn() you're bypassing shell expansion, and your child process is going to treat dots and asterisks in arguments literally. Try using sh child_process in place of child_process and see if you get better results.
Currently I'm failing to fetch the rsync output when I'm calling nodejs child_process.exec with a callback-function like in this snippet:
var sys = require('sys'),
exec = require('child_process').exec;
cmd = 'rsync -rpz test/test-files/one.txt jloos#test.mygnia.de:~/remote-test/a/b/'
exec(cmd, function(error, stdio, stderr) {
sys.print('s: ' + stdio + '\n');
sys.print('e: ' + stderr + '\n');
});
I think this is caused by the specific behavior of rsync. rsync communicates with it's counterpart via terminal. So how can I fetch the messages from rsync, if even possible?
When I use cmd = 'ls -la' I get the expected output.
Thanks
Often stdout is buffered when the program isn't running in a virtual terminal.
Many languages have a pty module which will trick the program into behaving as though it is running in a terminal.
This provides that functionality for NodeJs;
https://github.com/chjj/pty.js
Keep in mind that rsync may be writing lots of special characters or using something like ncurses to provide the updating status messages, which may make it more difficult to work with the output.
I want to execute a command like "doSomething ./myfiles/*.csv" with spawn in node.js. I want to use spawn instead of exec, because it is some kind of watch process and I need the stdout output.
I tried this
var spawn = require('child_process').spawn;
spawn("doSomething", ["./myfiles/*.csv"]);
But then the wildcard *.csv will not interpreted.
Is it not possible to use wildcards when using spawn()? Are there other possibilities to solve this problem?
Thanks
Torben
The * is being expanded by the shell, and for child_process.spawn the arguments are coming through as strings so will never get properly expanded. It's a limitation of spawn. You could try child_process.exec instead, it will allow the shell to expand any wildcards properly:
var exec = require("child_process").exec;
var child = exec("doSomething ./myfiles/*.csv",function (err,stdout,stderr) {
// Handle result
});
If you really need to use spawn for some reason perhaps you could consider expanding the wildcard file pattern yourself in Node with a lib like node-glob before creating the child process?
Update
In the Joyent Node core code we can observe an approach for invoking an arbitrary command in a shell via spawn while retaining full shell wildcard expansion:
https://github.com/joyent/node/blob/937e2e351b2450cf1e9c4d8b3e1a4e2a2def58bb/lib/child_process.js#L589
And here's some pseudo code:
var child;
var cmd = "doSomething ./myfiles/*.csv";
if ('win32' === process.platform) {
child = spawn('cmd.exe', ['/s', '/c', '"' + cmd + '"'],{windowsVerbatimArguments:true} );
} else {
child = spawn('/bin/sh', ['-c', cmd]);
}
Here's the simplest solution:
spawn("doSomething", ["./myfiles/*.csv"], { shell: true });
As #JamieBirch suggested in his comment, the key is telling spawn() to use the shell ({ shell: true }, see the docs), so the wildcard is properly resolved.
What OS are you using? In Unix-family OSs (e.g. Linux, MacOS), programs expect the shell process to expand wildcard filename arguments and pass the expansion in argv[]. In Windows OSs, programs usually expect to have to expand wildcards themselves (though only if they're Windows-native programs; ported Unix-family programs may at most try to run the arguments through a compatibility layer).
Your syntax looks like it's for a Unix-family system. If so, then when you call spawn() you're bypassing shell expansion, and your child process is going to treat dots and asterisks in arguments literally. Try using sh child_process in place of child_process and see if you get better results.
i want to implement a node.js program to checkup the filesystems(such as ext3..)status. However, the fs module only provide the operations of file. Must i use something else third part module?
One option would be to capture the output of the 'df' command and parse it.
You can run commands using child processes.
http://nodejs.org/docs/latest/api/child_processes.html#child_process.exec
var child_process = require('child_process');
child_process.exec('df', function(err, stdout, stderr) {
// 'stdout' here is a string containing the things printed by 'df'
console.log(stdout);
});