Say I want to run the following bash command: ./someCommand &. When I use the following Node:
var execute = require('child_process').exec;
execute(cmd, function (err, out) {
// Do stuff
});
I can never go inside the callback, and I think the problem is because the process is waiting for ./someCommand to end, but I've tried to fork it!! What do I do?
execute.('-cs -Some Command-', func(err, res){
//Do staff
})
cs - command shell.
And you can use spawn instead exec.
https://nodejs.org/api/child_process.html#child_process_child_process_spawn_command_args_options
If you don't want to wait, just put your code outside, after execute.
Related
I am looking for a way to pass or output data from one script to another script so that the late script can execute itself with the output that came from the first one.
Basically, I have a ruby script with some instructions in it and I want to pass (or output...) the result of the ruby script to a node.js script.
I would like help ( and examples ... ) on how to realize this and/or recommendations for techniques or technologies I might have never heard of it that might do the trick
Thank you.
You can use child_process exec to execute a script and handle it's output.
Ruby Script
# example.rb
puts "hello world"
Node Script
// example.js
const exec = require('child_process').exec
exec('ruby example.rb', function(err, stdout, stderr) {
console.error(err)
console.error('stderr: ' + stderr)
console.log('stdout: ' + stdout) // logs "hello world"
});
I have been using shelljs
On my super fast system I execute this:
var shell = require('shelljs')
const exec = require('child_process').exec
console.time('shell mktemp -d')
shell.exec('mktemp -d', {silent: true})
console.timeEnd('shell mktemp -d')
console.time('child exec mktemp -d')
exec('mktemp', ['-d'], function(error, stdout, stderr) {
if (error) {
console.error('stderr', stderr)
throw error
}
console.log('exec stdout', stdout)
console.timeEnd('child exec mktemp -d')
})
Its giving the following execution times:
shell mktemp -d: 208.126ms
exec stdout /tmp/tmp.w22tyS5Uyu
child exec mktemp -d: 48.812ms
Why is shelljs 4 times slower? Any thoughts?
Your code example compares async child_process.exec() with sync shell.exec(), which isn't entirely a fair comparison. I think you'll find shell.exec(..., { async: true }) performs a bit better: this is because sync shell.exec() does extra work to provide real-time stdio while still capturing stdout/stderr/return code as part of its return value; async shell.exec() can provide the same feature mostly for free.
Even with { silent: true }, the extra work is still necessary. shell.exec() is built on top of child_process.execSync(), which only returns stdout. We need to perform the same extra work in order to return return code and stderr.
Have a look to how shelljs is implemented:
It fully relies on node.js fs library. This library is cross platform and written in C++ but not as performant as C language. More generally, you can't have in JS the perfs you get in C...
Another thing, abstraction layers:
you're using exec(Command) where Command is a C tailored (Linux C here I think). The machine creates a thread and executes a command in it.
When using shell.js, there are many mechanisms to ensure cross plateform and keep the abstraction of your command as a function and keep the result as a variable. See the code of exec in shell.js:
https://github.com/shelljs/shelljs/blob/master/src/exec.js
It is not really doing the same thing as your line of code.
Hope that helps!
I want to execute a command like "doSomething ./myfiles/*.csv" with spawn in node.js. I want to use spawn instead of exec, because it is some kind of watch process and I need the stdout output.
I tried this
var spawn = require('child_process').spawn;
spawn("doSomething", ["./myfiles/*.csv"]);
But then the wildcard *.csv will not interpreted.
Is it not possible to use wildcards when using spawn()? Are there other possibilities to solve this problem?
Thanks
Torben
The * is being expanded by the shell, and for child_process.spawn the arguments are coming through as strings so will never get properly expanded. It's a limitation of spawn. You could try child_process.exec instead, it will allow the shell to expand any wildcards properly:
var exec = require("child_process").exec;
var child = exec("doSomething ./myfiles/*.csv",function (err,stdout,stderr) {
// Handle result
});
If you really need to use spawn for some reason perhaps you could consider expanding the wildcard file pattern yourself in Node with a lib like node-glob before creating the child process?
Update
In the Joyent Node core code we can observe an approach for invoking an arbitrary command in a shell via spawn while retaining full shell wildcard expansion:
https://github.com/joyent/node/blob/937e2e351b2450cf1e9c4d8b3e1a4e2a2def58bb/lib/child_process.js#L589
And here's some pseudo code:
var child;
var cmd = "doSomething ./myfiles/*.csv";
if ('win32' === process.platform) {
child = spawn('cmd.exe', ['/s', '/c', '"' + cmd + '"'],{windowsVerbatimArguments:true} );
} else {
child = spawn('/bin/sh', ['-c', cmd]);
}
Here's the simplest solution:
spawn("doSomething", ["./myfiles/*.csv"], { shell: true });
As #JamieBirch suggested in his comment, the key is telling spawn() to use the shell ({ shell: true }, see the docs), so the wildcard is properly resolved.
What OS are you using? In Unix-family OSs (e.g. Linux, MacOS), programs expect the shell process to expand wildcard filename arguments and pass the expansion in argv[]. In Windows OSs, programs usually expect to have to expand wildcards themselves (though only if they're Windows-native programs; ported Unix-family programs may at most try to run the arguments through a compatibility layer).
Your syntax looks like it's for a Unix-family system. If so, then when you call spawn() you're bypassing shell expansion, and your child process is going to treat dots and asterisks in arguments literally. Try using sh child_process in place of child_process and see if you get better results.
I want to execute a command like "doSomething ./myfiles/*.csv" with spawn in node.js. I want to use spawn instead of exec, because it is some kind of watch process and I need the stdout output.
I tried this
var spawn = require('child_process').spawn;
spawn("doSomething", ["./myfiles/*.csv"]);
But then the wildcard *.csv will not interpreted.
Is it not possible to use wildcards when using spawn()? Are there other possibilities to solve this problem?
Thanks
Torben
The * is being expanded by the shell, and for child_process.spawn the arguments are coming through as strings so will never get properly expanded. It's a limitation of spawn. You could try child_process.exec instead, it will allow the shell to expand any wildcards properly:
var exec = require("child_process").exec;
var child = exec("doSomething ./myfiles/*.csv",function (err,stdout,stderr) {
// Handle result
});
If you really need to use spawn for some reason perhaps you could consider expanding the wildcard file pattern yourself in Node with a lib like node-glob before creating the child process?
Update
In the Joyent Node core code we can observe an approach for invoking an arbitrary command in a shell via spawn while retaining full shell wildcard expansion:
https://github.com/joyent/node/blob/937e2e351b2450cf1e9c4d8b3e1a4e2a2def58bb/lib/child_process.js#L589
And here's some pseudo code:
var child;
var cmd = "doSomething ./myfiles/*.csv";
if ('win32' === process.platform) {
child = spawn('cmd.exe', ['/s', '/c', '"' + cmd + '"'],{windowsVerbatimArguments:true} );
} else {
child = spawn('/bin/sh', ['-c', cmd]);
}
Here's the simplest solution:
spawn("doSomething", ["./myfiles/*.csv"], { shell: true });
As #JamieBirch suggested in his comment, the key is telling spawn() to use the shell ({ shell: true }, see the docs), so the wildcard is properly resolved.
What OS are you using? In Unix-family OSs (e.g. Linux, MacOS), programs expect the shell process to expand wildcard filename arguments and pass the expansion in argv[]. In Windows OSs, programs usually expect to have to expand wildcards themselves (though only if they're Windows-native programs; ported Unix-family programs may at most try to run the arguments through a compatibility layer).
Your syntax looks like it's for a Unix-family system. If so, then when you call spawn() you're bypassing shell expansion, and your child process is going to treat dots and asterisks in arguments literally. Try using sh child_process in place of child_process and see if you get better results.
I'm trying to wrap the lftp program in a node.js application, using child_process. The problem is that lftp doesn't write its output to stdout, so I cannot catch its output in node.js. Sample code:
var proc = require('child_process').spawn('lftp', ['-p', port, '-u', username + ',' + password, host]);
proc.stdout.on('data', function (data) {
console.log('stdout:', data.toString('utf-8'));
});
proc.on('exit', function (code) {
console.log('process exited with code ' + code);
});
proc.stdin.write('ls');
// proc.stdin.end();
If I uncomment the line that calls stdin.end() for the lftp child process, the output from the ls command appears in my terminal as it should. If I don't the process simply hangs and nothing gets outputted.
I've also tried using unbuffer, but it doesn't seem to allow me to write to lftp's stdin anymore. It outputs the usual "[Resolving host address...]" stuff, but not the output from the ls command.
My question is: what do I have to do to be able to interact with lftp using node.js' child_process?
Well, this was dumb. I forgot to write a newline after the ls command to stdin. It seems to work without the need for unbuffer.