I've got some experience with Ruby and Rake, but now I'm working on a Node project and want to learn how to do the same things with Jake.
Ruby has a system function that will shell out to a command and wait for it to exit. Rake extends this by adding an sh function that will additionally throw an error if the child process returned a nonzero exit code (or couldn't be found at all). sh is really handy for Rake tasks that shell out to things like compilers or test frameworks, because it automatically terminates the task as soon as anything fails.
Node doesn't seem to have anything like system or sh -- it looks like the nearest equivalents are child_process.spawn and child_process.exec, but neither of them wires up STDOUT or STDERR, so you can't see any output from the child process unless you do some extra work.
What's the best way to get an sh method for Jake? (Though since this is Node, I'd expect it to be async, rather than blocking until the command returns like Ruby does.) Is there an npm module that has already invented this particular wheel, or does someone have a code sample that does this?
I've already seen sh.js, but it looks awfully heavyweight for this (it tries to build an entire command interpreter in Node), and it doesn't look like it's async (though the docs don't say one way or the other).
I'm looking for something that I could use more or less like this (using Jake's support for async tasks):
file('myprogram', ['in.c'], function() {
// sh(command, args, successCallback)
sh('gcc', ['in.c', '-o', 'myprogram'], function() {
// sh should throw if gcc couldn't be found or returned nonzero.
// So if we got here, we can tell Jake our task completed successfully.
complete();
});
}, true);
Here's some code I've come up with that seems to work well. (But if anyone has a better answer, or knows of an existing npm module that already does this, please add another answer.)
Supports full shell syntax, so you can use | and < and > to pipe and redirect output, you can run Windows batch files, etc.
Displays output (both STDOUT and STDERR) as the child process generates it, so you see incremental output as the command runs.
No limitation on the amount of output the command can generate (unlike a previous exec-based version of this code).
Cross-platform (works on Windows, should work on Mac/Linux as well). I borrowed the platform-specific-shell (if platform === 'win32') technique from npm.
Here's the code:
function sh(command, callback) {
var shell = '/bin/sh', args = ['-c', commandLine], child;
if (process.platform === 'win32') {
shell = 'cmd';
args = ['/c', commandLine];
}
child = child_process.spawn(shell, args);
child.stdout.pipe(process.stdout);
child.stderr.pipe(process.stderr);
child.on('exit', function(code, signal) {
if (signal != null)
throw new Error("Process terminated with signal " + signal);
if (code !== 0)
throw new Error("Process exited with error code " + code);
callback();
});
};
Related
I'm facing a very odd issue where I have a Node script which invokes a process, it looks like this:
// wslPath declared here (it's a shell file)
const proc = cp.spawn('ubuntu.exe', ['run', wslPath]);
let stdout = '';
proc.stdout.on('data', data => stdout += data.toString());
let stderr = '';
proc.stderr.on('data', data => stderr += data.toString());
return await new Promise((resolve, reject) => {
proc.on('exit', async code => {
await fs.remove(winPath);
if (code) {
reject({ code, stdout, stderr });
}
resolve({ stdout, stderr });
});
});
As you can see, the script invokes WSL. WSL is enabled on the computer. When I run this script manually, it works fine. When I log in to the computer the script is at using RDP from another computer and run it with the same credentials, it works fine as well. But when the script is invoked from a scheduled task which also runs with the same credentials, the spawn call returns:
(node:4684) UnhandledPromiseRejectionWarning: Error: spawn UNKNOWN
at ChildProcess.spawn (internal/child_process.js:394:11)
at Object.spawn (child_process.js:540:9)
I verified the user is the same by logging require('os').userInfo() and require('child_process').spawnSync('whoami', { encoding: 'utf8' }) and it returns the same in all three cases.
I assume it is because ubuntu.exe is not being found, but I don't know why that would be as the user is the same in all three cases.
What could be the reason for this and how can I debug this further?
The Windows Task Scheduler allows you to specify a user to run as (for privilege reasons), but does not give you the environment (PATH and other environment variables) that are configured for that user.
So, when running programs from the Windows Task Scheduler, it's important to not make any assumptions about what's in the environment (particularly the PATH). If my program depends on certain things in the environment, I will sometimes change my Task to be a .BAT file that first sets up the environment as needed and then launch my program from there.
Among other things, the simplest way to not rely on the path is to specify the full path to the executable you are running rather than assuming it will be found in the path somewhere. But, you also need to make sure that your executable can find any other resources it might need without any environment variables or you need to configure those environment variables for it before running.
I'm running the following code using node and I always get only about 80% of what stdout is supposed to return when the file is past 12ko or so. If the file is 240ko it will still output 80% of it. If it's under 12 it will output completely.
When I open a cmd and run the command manually I always get the full output.
I tried exec, execFile, I tried increasing the max buffer or changing the encoding and it's not the issue. I tried to add options {shell: true, detached: true} but it vain, in fact, when I run it as detached it seems to be running completely fine as it does open an actual console but I'm not able to retrieve the stdout when the process is completed.
const spawn = require('child_process').spawn;
const process = spawn(
'C:\\Users\\jeanphilipped\\Desktop\\unrtf\\test\\unrtf.exe',
['--html' ,'C:\\Users\\jeanphilipped\\Desktop\\unrtf\\test\\tempaf45.rtf'],
);
let chunks = [];
process.stdout.on('data', function (msg) {
chunks = [...chunks, ...msg];
});
process.on('exit', function (msg) {
const buffer = Buffer.from(chunks);
console.log(buffer.toString());
});
Any clues ? It seems to be Node since when I run it manually everything works fine.
according to nodejs documentation, all of the child_process commands are asynchronous . so when you try to access your chunk variable there is no guarantee that you command has been finished , maybe your command is still on process. so it is recommended you should wrap your entire child_process command in async/await function.
I'm trying to make this little QoL app that can trigger my scripts inside package.json of any project I throw at it (react, polymer, etc.).
So far, it works as it should. At least until I kill the app or want to terminate the specific process (like using ctrl+c while running a npm start in a console).
I'm currently calling the command this way:
const exec = require("child_process").execFile;
...
let ps = exec("command.bat", [path, command], (error, stdout, stderr)=>{}) //command.bat contains only: cd "%1" \n npm "%2"
Previously I've used node-powershell like so:
const Shell = require('node-powershell');
...
let sh = new Shell();
sh.addCommand(`cd ${fs.realpathSync(path)}`);
sh.addCommand(`npm ${command}`);
sh.invoke();
And I've already tried using ps-tree in hopes that this will list all the processes started by my process so I can kill them but no luck and because it creates additional process or two it's getting out of hands really quickly.
const cp = require('child_process');
...
psTree(ps.pid, function (err, children) {
cp.spawn('kill', ['-9'].concat(children.map(function (p) { return p.PID })));
});
So if there is some solution I would be really grateful. I'm also open to any different solution if there is any.
Thanks in advance.
I am writing a CLI tool for a node.js app. Some of the commands have to run npm and show the results. This is what I have so far:
import {spawn} from 'child_process';
let projectRoot = '...';
let npm = (process.platform === "win32" ? "npm.cmd" : "npm"),
childProcess = spawn(npm, ["install"], { cwd: projectRoot });
childProcess.stdout.pipe(process.stdout);
childProcess.stderr.pipe(process.stderr);
childProcess.on('close', (code) => {
// Continue the remaining operations
});
The command does run fine and outputs the results (or errors). However, it doesn't give me a live feed with the progress bar, etc. It waits until the entire operation is over and then dumps the output into the console.
I've tried different variations of the spawn configuration but I can't get it to show me the live feed.
I am on Windows 10 and use node.js 4 and npm 3.
As discussed in the comments: Run the spawn with { stdio: 'inherit' }.
However, good question is why the 'manual piping' does not do the same. I think that's because npm uses the 'fancy progress bar'. It probably uses some special way how to deal with stdout that does not play well with process.stdout. If you try some other long-running command (such as 'find ./'), your way of piping works fine.
I was recently going to test out running phantomJS from python as a commandline argument, I haven't got round to it yet but have seen examples. Because PhantomJS is run from the command line this seems to be possible. The result that PhantomJS would spit out would go straight into a variable.
Before I go down that path, making this work in node.js would actually be more useful for me and it got me thinking, can i just use to node to run PhantomJS as a program gets run from the commandline and store the data result that PhantomJS would normally spit out into a variable?
I would rather not use phantomjs-node because it seems to be using too many tricks.
The reason for all of this is to be able to run PhantomJS at the same time as another action the program takes and use the resulting data its recorded for some other stuff.
Simply put, you can run system command line stuff in python, can I do the same in node.js?
Cheers :)
Edit: I understand that node and phantom use different js environments, that's cool because I just want to run phantom as its own process and catch all that output data into a node.js variable (the data will be a array of a pair, string and floating point.) I don't want to 'drive' with phantom, I will craft the loaded javascript files todo what I want. All I want is phantom output. :)
From NPM: https://npmjs.org/package/phantomjs
var path = require('path')
var childProcess = require('child_process')
var phantomjs = require('phantomjs')
var binPath = phantomjs.path
var childArgs = [
path.join(__dirname, 'phantomjs-script.js'),
'some other argument (passed to phantomjs script)'
]
childProcess.execFile(binPath, childArgs, function(err, stdout, stderr) {
// handle results
})
I suppose you can make a simple script for Node.js to run; in that script phantomjs script will be run as a child process. You can see the working example (and links for some documentation) in this answer. I suppose this discussion might be helpful for you as well.
As an alternative to Donald Derek's answer, you can use the spawn function. It will allow you to read the child process's output as soon as it's produced rather than the output being buffered and returned to you all at once.
You can read more about it here.
An example from the documentation.
var spawn = require('child_process').spawn,
ls = spawn('ls', ['-lh', '/usr']);
ls.stdout.on('data', function (data) {
console.log('stdout: ' + data);
});
ls.stderr.on('data', function (data) {
console.log('stderr: ' + data);
});
ls.on('close', function (code) {
console.log('child process exited with code ' + code);
});