Say I have a named pipe on linux:
mkfifo lk.log
From the command line, I can do this to print out anything written to the name pipe file.
node monitor.js < lk.log
and pretend this is what the script looks like
// monitor.js
process.stdin.resume();
process.stdin.setEncoding('utf8');
// read data from stdin
process.stdin.on('data', function(chunk) {
console.log(chunk);
});
How could I do this within node using child_process.spawn?
child_process.spawn('node', ['monitor.js'])...
The easiest way would be to use exec():
var exec = require('child_process').exec;
exec('node monitor.js < lk.log', function(err, stdout, stderr) {
...
});
A more elaborate way would be to open the named pipe in node and pass it as stdin to the process you're spawning (see the option stdio for spawn).
The answer is to use fs.open and the stdio options in child_process.spawn as such:
var spawn = require('child_process').spawn;
var fd_stdin = fs.openSync('lk.log', 'r');
spawn('node', ['monitor.js'], {
stdio: [fd_stdin, 1, 2];
});
From Ben Noordhuis (Core Node contributor) - 10/11/11
Windows has a concept of named pipes but since you mention mkfifo I
assume you mean UNIX FIFOs.
We don't support them and probably never will (FIFOs in non-blocking
mode have the potential to deadlock the event loop) but you can use
UNIX sockets if you need similar functionality.
https://groups.google.com/d/msg/nodejs/9TvDwCWaB5c/udQPigFvmgAJ
For unix sockets, see: https://stackoverflow.com/a/18226566/977939
Related
As an exercise, I'm trying to use a systemcall from node.js to write a small text file to the /tmp directory. Here is my code:
#!/bin/node
var child_process = require("child_process");
var send = "Hello, world!";
child_process.exec('cat - > /tmp/test1', { input: send });
The file actually gets created; but, no content is placed in it. Things just hang. Can someone please tell me what I'm missing?
Also, I'd really like to know how to do this synchronously.
Thanks for any input.
... doug
hm unless i forgot to rtm too, this code will just never work. There is no such input option for cp.exec.
But there is a stdio option, will let us open the expected stdio on the child.
child_process.exec('cat - > /tmp/test1', { stdio: 'pipe' });
see https://nodejs.org/api/child_process.html#child_process_options_stdio
stdios are not string, they are streams, which we can end / write / pipe / close / push etc
see https://nodejs.org/api/stream.html
Note that stdin is a writable, stdout / stderr are readable.
To write the stdin of cat you ll now consume the cp.stdin object and call for its end() method.
child_process.exec('cat - > /tmp/test1', { stdio: 'pipe' }).stdin.end('hello world');
Note that end method is a write followed by a termination of the stream, which is required to tell cat to quit.
To ensure this is working well, we should refactor it, to not send stdin to a file, instead pipe child.stdout to the process.stdout.
var child_process = require('child_process');
var cp = child_process.exec('cat -', { stdio: 'pipe' });
cp.stdin.end('hello world');
cp.stdout.pipe(process.stderr);
Note that process is a global.
I finally got my original approach to work. The big stumbling block is to know that the synchronous methods are only available in version 0.12 (and later) of node.js. Here is the code that I finally got to work:
#!/usr/local/n/versions/node/0.12.14/bin/node
var child_process = require('child_process');
var send = "Hello, world!"
child_process.execSync('cat - > /tmp/test1', { input : send }).toString();
Thanks to all for the help.
... doug
I'd like to use the execSync method which was added in NodeJS 0.12 but still have the output in the console window from which i ran the Node script.
E.g. if I run a NodeJS script which has the following line I'd like to see the full output of the rsync command "live" inside the console:
require('child_process').execSync('rsync -avAXz --info=progress2 "/src" "/dest"');
I understand that execSync returns the ouput of the command and that I could print that to the console after execution but this way I don't have "live" output...
You can pass the parent´s stdio to the child process if that´s what you want:
require('child_process').execSync(
'rsync -avAXz --info=progress2 "/src" "/dest"',
{stdio: 'inherit'}
);
You can simply use .toString().
var result = require('child_process').execSync('rsync -avAXz --info=progress2 "/src" "/dest"').toString();
console.log(result);
Edit: Looking back on this, I've realised that it doesn't actually answer the specific question because it doesn't show the output to you 'live' — only once the command has finished running.
However, I'm leaving this answer here because I know quite a few people come across this question just looking for how to print the result of the command after execution.
Unless you redirect stdout and stderr as the accepted answer suggests, this is not possible with execSync or spawnSync. Without redirecting stdout and stderr those commands only return stdout and stderr when the command is completed.
To do this without redirecting stdout and stderr, you are going to need to use spawn to do this but it's pretty straight forward:
var spawn = require('child_process').spawn;
//kick off process of listing files
var child = spawn('ls', ['-l', '/']);
//spit stdout to screen
child.stdout.on('data', function (data) { process.stdout.write(data.toString()); });
//spit stderr to screen
child.stderr.on('data', function (data) { process.stdout.write(data.toString()); });
child.on('close', function (code) {
console.log("Finished with code " + code);
});
I used an ls command that recursively lists files so that you can test it quickly. Spawn takes as first argument the executable name you are trying to run and as it's second argument it takes an array of strings representing each parameter you want to pass to that executable.
However, if you are set on using execSync and can't redirect stdout or stderr for some reason, you can open up another terminal like xterm and pass it a command like so:
var execSync = require('child_process').execSync;
execSync("xterm -title RecursiveFileListing -e ls -latkR /");
This will allow you to see what your command is doing in the new terminal but still have the synchronous call.
Simply:
try {
const cmd = 'git rev-parse --is-inside-work-tree';
execSync(cmd).toString();
} catch (error) {
console.log(`Status Code: ${error.status} with '${error.message}'`;
}
Ref: https://stackoverflow.com/a/43077917/104085
// nodejs
var execSync = require('child_process').execSync;
// typescript
const { execSync } = require("child_process");
try {
const cmd = 'git rev-parse --is-inside-work-tree';
execSync(cmd).toString();
} catch (error) {
error.status; // 0 : successful exit, but here in exception it has to be greater than 0
error.message; // Holds the message you typically want.
error.stderr; // Holds the stderr output. Use `.toString()`.
error.stdout; // Holds the stdout output. Use `.toString()`.
}
When command runs successful:
Add {"encoding": "utf8"} in options.
execSync(`pwd`, {
encoding: "utf8"
})
I'm trying to execute Inkscape by passing data via stdin. Inkscape only supports this via /dev/stdin. Basically, I'm trying to do something like this:
echo "<sgv>...</svg>" | inkscape -z -f /dev/stdin -A /dev/stdout
I don't want to have to write the SVG to disk.
I tried just using stdin.write(), but it doesn't work (maybe because of /dev/stdin):
var cmd = spawn("inkscape", ["-z", "-f", "/dev/stdin", "-A", "/dev/stdout"], {encoding: "buffer", stdio: ["pipe", stdoutPipe, "pipe"]});
cmd.stdin.write(svg);
This does work, but I have to write the SVG to disk:
var cmd = spawn("inkscape", ["-z", "-f", "/dev/stdin", "-A", "/dev/stdout"], {encoding: "buffer", stdio: [fs.openSync('file.svg', "a"), stdoutPipe, "pipe"]});
I tried passing a stream to stdio, but I just keep getting TypeError: Incorrect value for stdio stream: [object Object]
Any ideas?
Addendum
The examples use Inkscape, but my question applies to any arbitrary program using /dev/stdin.
By the way, this would work for me:
var exec = require('child_process').exec;
exec("echo \"<svg>...</svg>\" | inkscape -z -f /dev/stdin -A /dev/stdout | cat", function (error, stdout, stderr) {});
Except, my SVG is too long, so it throws an error: Error: spawn Unknown system errno 7
Alright, I don't have Inkscape, but this appears to solve the Node.js side of things. I'm using wc as a stand in Inkscape; the -c option simply outputs the number of bytes in a given file (in this case /dev/stdin).
var child_process = require('child_process');
/**
* Create the child process, with output piped to the script's stdout
*/
var wc = child_process.spawn('wc', ['-c', '/dev/stdin']);
wc.stdout.pipe(process.stdout);
/**
* Write some data to stdin, and then use stream.end() to signal that we're
* done writing data.
*/
wc.stdin.write('test');
wc.stdin.end();
The trick seems to be signaling that you're done writing to the stream. Depending on how large your SVG is, you may need to pay attention to backpressure from Inkscape by handling the 'drain' event.
As for passing a stream into the child_process.spawn call, you instead need to use the 'pipe' option, and then pipe a readable stream into child.stdin, as shown below. I know this works in Node v0.10.26, but not sure about before that.
var stream = require('stream');
var child_process = require('child_process');
/**
* Create the child process, with output piped to the script's stdout
*/
var wc = child_process.spawn('wc', ['-c', '/dev/stdin'], {stdin: 'pipe'});
wc.stdout.pipe(process.stdout);
/**
* Build a readable stream with some data pushed into it.
*/
var readable = new stream.Readable();
readable._read = function noop() {}; // See note below
readable.push('test me!');
readable.push(null);
/**
* Pipe our readable stream into wc's standard input.
*/
readable.pipe(wc.stdin);
Obviously, this method is a bit more complicated, and you should use the method above unless you have good reason to (you're effectively implementing your own readable string).
Note: The readable._push function must be implemented according to the docs, but it doesn't necessarily have to do anything.
So, I figured out a work around. This seems like a bit of a hack, but it works just fine.
First, I made this one line shell script:
cat | inkscape -z -f /dev/stdin -A /dev/stdout | cat
Then, I simply spawn that file and write to the stdin like this:
cmd = spawn("shell_script");
cmd.stdin.write(svg);
cmd.stdin.end();
cmd.stdout.pipe(pipe);
I really think this should work without the shell script, but it won't (for me at least). This may be a Node.js bug.
The problem comes from the fact that file descriptors in node are sockets and that linux (and probably most Unices) won't let you open /dev/stdin if it's a socket.
I found this explanation by bnoordhuis on https://github.com/nodejs/node-v0.x-archive/issues/3530#issuecomment-6561239
The given solution is close to #nmrugg's answer :
var run = spawn("sh", ["-c", "cat | your_command_using_dev_stdin"]);
After further work, you can now use the https://www.npmjs.com/package/posix-pipe module to make sure that the process sees a stdin that is not a socket.
look at the 'should pass data to child process' test in this module which boils down to
var p = pipe()
var proc = spawn('your_command_using_dev_stdin', [ .. '/dev/stdin' .. ],
{ stdio: [ p[0], 'pipe', 'pipe' ] })
p[0].destroy() // important to avoid reading race condition between parent/child
proc.stdout.pipe(destination)
source.pipe(p[1])
As Inkscape bug 171016 indicates, Inkscape does not support importing via stdin, but it is on their Wishlist.
Let's say I have a file "/tmp/sample.txt" and I want to move it to "/var/www/mysite/sample.txt" which is in a different volume.
How can i move the file in node.js?
I read that fs.rename only works inside the same volume and util.pump is already deprecated.
What is the proper way to do it? I read about stream.pipe, but I couldn't get it to work. A simple sample code would be very helpful.
Use the mv module:
var mv = require('mv');
mv('source', 'dest', function(err) {
// handle the error
});
If on Windows and don't have 'mv' module, can we do like
var fs = require("fs"),
source = fs.createReadStream("c:/sample.txt"),
destination = fs.createWriteStream("d:/sample.txt");
source.pipe(destination, { end: false });
source.on("end", function(){
fs.unlinkSync("C:/move.txt");
});
The mv module, like jbowes stated, is probably the right way to go, but you can use the child process API and use the built-in OS tools as an alternative. If you're in Linux use the "mv" command. If you're in Windows, use the "move" command.
var exec = require('child_process').exec;
exec('mv /temp/sample.txt /var/www/mysite/sample.txt',
function(err, stdout, stderr) {
// stdout is a string containing the output of the command.
});
You can also use spawn if exec doesn't work properly.
var spawn = require("child_process").spawn;
var child = spawn("mv", ["data.csv","./done/"]);
child.stdout.on("end", function () {
return next(null,"finished")
});
Hope this helps you out.
I'm trying to launch a shell command from Node.js, without redirecting that command's input and output -- just like shelling out to a command using a shell script, or using Ruby's system command. If the child process wants to write to STDOUT, I want that to go straight to the console (or get redirected, if my Node app's output was redirected).
Node doesn't seem to have any straightforward way to do this. It looks like the only way to run another process is with child_process, which always redirects the child process's input and output to pipes. I can write code to accept data from those pipes and write it to my process's STDOUT and STDERR, but if I do that, the APIs force me to sacrifice some flexibility.
I want two features:
Shell syntax. I want to be able to pipe output between commands, or run Windows batch files.
Unlimited output. If I'm shelling out to a compiler and it wants to generate megabytes of compiler warnings, I want them all to scroll across the screen (until the user gets sick of it and hits Ctrl+C).
It looks like Node wants to force me choose between those two features.
If I want an unlimited amount of output, I can use child_process.spawn and then do child.stdout.on('data', function(data) { process.stdout.write(data); }); and the same thing for stderr, and it'll happily pipe data until the cows come home. Unfortunately, spawn doesn't support shell syntax.
If I want shell syntax, I can use child_process.exec. But exec insists on buffering the child process's STDOUT and STDERR for me and giving them to me all at the end, and it limits the size of those buffers (configurable, 200K by default). I can still hook the on('data') events, if I want to see the output as it's generated, but exec will still add the data to its buffers too. When the amount of data exceeds the predefined buffer size, exec will terminate the child process.
(There's also child_process.execFile, which is the worst of both worlds from a flexibility standpoint: no shell syntax, but you still have to cap the amount of output you expect.)
Am I missing something? Is there any way to just shell out to a child process in Node, and not redirect its input and output? Something that supports shell syntax and doesn't crap out after a predefined amount of output, just like is available in shell scripts, Ruby, etc.?
You can inherit stdin/out/error streams via spawn argument so you don't need to pipe them manually:
var spawn = require('child_process').spawn;
spawn('ls', [], { stdio: 'inherit' });
Use shell for shell syntax - for bash it's -c parameter to read script from string:
var spawn = require('child_process').spawn;
var shellSyntaxCommand = 'ls -l | grep test | wc -c';
spawn('sh', ['-c', shellSyntaxCommand], { stdio: 'inherit' });
To summarise:
var spawn = require('child_process').spawn;
function shspawn(command) {
spawn('sh', ['-c', command], { stdio: 'inherit' });
}
shspawn('ls -l | grep test | wc -c');
You can replace exec by spawn and use the shell syntax simply with:
const {spawn} = require ('child_process');
const cmd = 'ls -l | grep test | wc -c';
const p = spawn (cmd, [], {shell: true});
p.stdout.on ('data', (data) => {
console.log (data.toString ());
});
The magic is just {shell: true}.
I haven't used it, but I've seen this library: https://github.com/polotek/procstreams
It you'd do this. The .out() automatically pipes to the process's stdin/out.
var $p = require('procstreams');
$p('cat lines.txt').pipe('wc -l').out();
If doesn't support shell syntax, but that's pretty trivial I think.
var command_str = "cat lines.txt | wc -l";
var cmds = command_str.split(/\s?\|\s?/);
var cmd = $p(cmds.shift());
while(cmds.length) cmd = cmd.pipe(cmds.shift());
cmd
.out()
.on('exit', function() {
// Do whatever
});
There's an example in the node docs for the child_process module:
Example of detaching a long-running process and redirecting its output to a file:
var fs = require('fs'),
spawn = require('child_process').spawn,
out = fs.openSync('./out.log', 'a'),
err = fs.openSync('./out.log', 'a');
var child = spawn('prg', [], {
detached: true,
stdio: [ 'ignore', out, err ]
});
child.unref();