When creating a CLI I would like to test my function. For that I'm using the module child_process.
const path = require('path');
const { execFile } = require('child_process');
describe('cli test', () => {
test('thing', () => {
const myCli = execFile(
`${path.resolve(__dirname, '..')}/cli.js`, ['--foo', 'Bar'],
(err, stdout, stderr) => {
if (err) {
console.log('err: ', err);
}
});
});
But this produces the following error:
Attempted to log "err: { Error: spawn /projects/cli/src/cli.js EACCES
at Process.ChildProcess._handle.onexit (internal/child_process.js:240:19)
at onErrorNT (internal/child_process.js:415:16)
at process._tickCallback (internal/process/next_tick.js:63:19)
errno: 'EACCES',
code: 'EACCES',
Running this script directly in the terminal via the command: $ node cli.js --foo Bar works perfectly.
Now a suggestion is to chmod +x <file> that file (source). But the test should also work on CI, and on a different computer which pulls the Git repo.
Any idea?
I'd suggest using fork instead of execFile.
The child_process.fork() method is a special case of child_process.spawn() used specifically to spawn new Node.js processes.
This will allow you to execute JS files without needing them to be shell executable.
To the best of my knowledge, git actually tracks the executable bit for files. There are some things to consider though as pointed out in this article: https://medium.com/#tahteche/how-git-treats-changes-in-file-permissions-f71874ca239d
Another solution would be to not rely on the ./ execution syntax (which requires the executable bit to be turned on for the respective file) but instead to explicitly use the shell command:
const path = require('path');
const { execFile } = require('child_process');
describe('cli test', () => {
test('thing', () => {
const myCli = execFile(
`sh ${path.resolve(__dirname, '..')}/cli.js`, ['--foo', 'Bar'],
(err, stdout, stderr) => {
if (err) {
console.log('err: ', err);
}
});
});
Notice the sh prefix I added to your code, This way you thell the sh command (which should be available in all of your environments e.g. the CI) to execute the contents of the file, regardless of whether the file itself can be executed or not!
I was receiving an EACCESS -13 error from child_process.spawn when trying to run a the command line mysql command.
There was something wrong with my PATH and updating it to add /usr/local/mysql/bin/ resolved the problem.
The temporary fix is to run export PATH=$PATH:/usr/local/mysql/bin/.
The permanent fix is to:
type: sudo nano /etc/paths
Add /usr/local/mysql/bin at the end
Ctrl + X
Yes
Enter key
type hash -r # command line or close the terminal app and open it again
NOTE: I got the temporary fix from a site ... I don't know why it has a / on the end of the bin but all of the mysql executables appear to be available without it in the /etc/paths file
Related
Exiftool can not be spawned in aws lambda environment, getting below error:
ERROR { Error: spawn /var/task/node_modules/dist-exiftool/node_modules/exiftool.pl/vendor/exiftool ENOENT
at Process.ChildProcess._handle.onexit (internal/child_process.js:240:19)
at onErrorNT (internal/child_process.js:415:16)
at process._tickCallback (internal/process/next_tick.js:63:19)
errno: 'ENOENT',
code: 'ENOENT',
syscall:
'spawn /var/task/node_modules/dist-exiftool/node_modules/exiftool.pl/vendor/exiftool',
path:
'/var/task/node_modules/dist-exiftool/node_modules/exiftool.pl/vendor/exiftool',
spawnargs:
[ '-echo2', '1574158488325', '-stay_open', 'True', '-#', '-' ] }
A function is working properly with NodeJs version 8.10 on AWS Lamda but I want to upgrade this function to NodeJs 10.X version.
I failed to run a function on NodeJs 10.X. I am always getting an error on followings line.
const ep = new exiftool.ExiftoolProcess(exiftoolBin);
My function:
const exiftool = require('node-exiftool')
const exiftoolBin = require('dist-exiftool')
const fs = require('fs')
const path = require('path')
const ep = new exiftool.ExiftoolProcess(exiftoolBin)
const PHOTO_PATH = path.join(__dirname, 'photo.jpg')
const rs = fs.createReadStream(PHOTO_PATH)
ep.open()
.then(() => ep.readMetadata(rs, ['-File:all']))
.then((res) => {
console.log(res)
})
.then(() => ep.close(), () => ep.close())
.catch(console.error)
Please help me to resolve this issue.
Thank you.
The error message is rather misleading but there are some hints. The path says /exiftool.pl/, pl usually being perl. If you open the exiftool file you'll see #!/usr/bin/perl -w on the first line.
What's actually missing (ENOENT) is a perl binary
AWS apparently removed various binaries from the runtime environments. It seems the best practice now is using Layers to add missing dependencies.
So first, add perl to your AWS Lambda Function as a Layer, to do that you could use an ARN from https://metacpan.org/pod/AWS::Lambda. The perl binary should then be available at /opt/bin/perl, which doesn't match the #! of the script, so we have to adjust the function code:
const { ExiftoolProcess } = require('node-exiftool');
const exiftoolBin = require('dist-exiftool');
// Get perl binary path from environment variables or fallback to default.
const perlBin = process.env.PERL_BIN || '/opt/bin/perl';
// Instead of calling the perl script directly causing #! to choose the
// interpreter path, we'll pass the script path to the interpreter directly.
const exiftool = new ExiftoolProcess(`${perlBin} ${exiftoolBin}`);
But now the path passed to ExiftoolProcess is no longer a valid filepath. It's a shell command that is passed on to child_process.spawn(). Therefore we have to make sure it's handled as such by doing:
exiftool.open({ shell: true })
You might be done at this point, or you might run into the following error:
perl: error while loading shared libraries: libcrypt.so.1: cannot open shared object file: No such file or directory. I'm not sure how to best resolve this problem. Maybe compiling perl yourself and creating a layer on your own prevents it. But the quick and dirty solution that worked for me was copying the libcrypt.so.1 file from my local ubuntu system ($ whereis libcrypt.so.1), putting it into a folder called lib, zipping that folder and uploading it as a new layer.
I'm using GitBash in windows 10 and want to execute git commands within a child_process.exec call. I think that since I installed git through "Git For Windows", I just need specify the shell as the GitBash executable. I've tried every variation of the path to GitBash executable that I could think of and it always fails. What is the path that node is looking for?
Example of paths that do not work
c:/program files/git/usr/bin/bash
c:/program\ files/git/usr/bin/bash
/c/program\ files/git/usr/bin/bash
c:\\program files\\git\\usr\\bin\\bash
const { expect } = require('chai');
const { exec } = require('child_process');
describe.only('exec', function(){
it('should work', function(done){
let shellPath = "c:\\program files\\git\\usr\\bin\\bash";
expect(exec(`cat <<< "abc"`, { shell: shellPath }, (err, stdout) => {
expect(err).to.be.null;
expect(stdout.trim()).to.be.equal("abc");
done();
}));
});
});
The first assertion fails with:
expected [Error: Command failed: cat <<< "abc" << was unexpected at this time.] to be null
There are some problems with this approach.
As the reference states, exec automatically uses Windows-specific shell arguments that won't work for Bash.
Another problem is that PATH may not be set to GitBash binaries path.
This should likely work:
delete process.platform;
process.platform = 'linux';
exec(`cat <<< "abc"`, {
env: { PATH: 'C:\\Program Files\\git\\usr\\bin' },
shell: 'C:\\Program Files\\git\\usr\\bin\\bash.exe'
}, (err, stdout) => {
...
});
process.platform = 'win32';
The workability of this solution may depend on bash.exe implementation.
The use of custom shell isn't needed to run git in Node; this is handled by Git executable.
I need to run a shell script file using nodeJS that executes a set of Cassandra DB commands. Can anybody please help me on this.
inside db.sh file:
create keyspace dummy with replication = {'class':'SimpleStrategy','replication_factor':3}
create table dummy (userhandle text, email text primary key , name text,profilepic)
You could use "child process" module of nodejs to execute any shell commands or scripts with in nodejs. Let me show you with an example, I am running a shell script(hi.sh) with in nodejs.
hi.sh
echo "Hi There!"
node_program.js
const { exec } = require('child_process');
var yourscript = exec('sh hi.sh',
(error, stdout, stderr) => {
console.log(stdout);
console.log(stderr);
if (error !== null) {
console.log(`exec error: ${error}`);
}
});
Here, when I run the nodejs file, it will execute the shell file and the output would be:
Run
node node_program.js
output
Hi There!
You can execute any script just by mentioning the shell command or shell script in exec callback.
You can execute any shell command using the shelljs module
const shell = require('shelljs')
shell.exec('./path_to_your_file')
you can go:
var cp = require('child_process');
and then:
cp.exec('./myScript.sh', function(err, stdout, stderr) {
// handle err, stdout, stderr
});
to run a command in your $SHELL.
Or go
cp.spawn('./myScript.sh', [args], function(err, stdout, stderr) {
// handle err, stdout, stderr
});
to run a file WITHOUT a shell.
Or go
cp.execFile();
which is the same as cp.exec() but doesn't look in the $PATH.
You can also go
cp.fork('myJS.js', function(err, stdout, stderr) {
// handle err, stdout, stderr
});
to run a javascript file with node.js, but in a child process (for big programs).
EDIT
You might also have to access stdin and stdout with event listeners. e.g.:
var child = cp.spawn('./myScript.sh', [args]);
child.stdout.on('data', function(data) {
// handle stdout as `data`
});
Also, you can use shelljs plugin.
It's easy and it's cross-platform.
Install command:
npm install [-g] shelljs
What is shellJS
ShellJS is a portable (Windows/Linux/OS X) implementation of Unix
shell commands on top of the Node.js API. You can use it to eliminate
your shell script's dependency on Unix while still keeping its
familiar and powerful commands. You can also install it globally so
you can run it from outside Node projects - say goodbye to those
gnarly Bash scripts!
An example of how it works:
var shell = require('shelljs');
if (!shell.which('git')) {
shell.echo('Sorry, this script requires git');
shell.exit(1);
}
// Copy files to release dir
shell.rm('-rf', 'out/Release');
shell.cp('-R', 'stuff/', 'out/Release');
// Replace macros in each .js file
shell.cd('lib');
shell.ls('*.js').forEach(function (file) {
shell.sed('-i', 'BUILD_VERSION', 'v0.1.2', file);
shell.sed('-i', /^.*REMOVE_THIS_LINE.*$/, '', file);
shell.sed('-i', /.*REPLACE_LINE_WITH_MACRO.*\n/, shell.cat('macro.js'), file);
});
shell.cd('..');
// Run external tool synchronously
if (shell.exec('git commit -am "Auto-commit"').code !== 0) {
shell.echo('Error: Git commit failed');
shell.exit(1);
}
Also, you can use from the command line:
$ shx mkdir -p foo
$ shx touch foo/bar.txt
$ shx rm -rf foo
I'm on node v4.4.0 and on Windows 10. I'm using bunyan to log my node application.
try {
var fs = require('fs');
var path = require('path');
var spawn = require('child_process').spawn;
var through = require('through');
} catch (err) {
throw err;
}
var prettyStream = function () {
// get the binary directory of bunyan
var bin = path.resolve(path.dirname(require.resolve('bunyan')), '..', 'bin', 'bunyan');
console.log(bin); // this outputs C:\www\nodeapp\src\node_modules\bunyan\bin\bunyan, the file does exist
var stream = through(function write(data) {
this.queue(data);
}, function end() {
this.queue(null);
});
// check if bin var is not empty and that the directory exists
if (bin && fs.existsSync(bin)) {
var formatter = spawn(bin, ['-o', 'short'], {
stdio: [null, process.stdout, process.stderr]
});
// stream.pipe(formatter.stdin); // <- did this to debug
}
stream.pipe(process.stdout); // <- did this to debug
return stream;
}
The logging spits out in the console due to the fact I used stream.pipe(process.stdout);, i did this to debug the rest of the function.
I however receive the error:
Error: spawn C:\www\nodeapp\src\node_modules\bunyan\bin\bunyan ENOENT
at exports._errnoException (util.js:870:11)
at Process.ChildProcess._handle.onexit (internal/child_process.js:178:32)
at onErrorNT (internal/child_process.js:344:16)
at nextTickCallbackWith2Args (node.js:442:9)
at process._tickCallback (node.js:356:17)
at Function.Module.runMain (module.js:443:11)
at startup (node.js:139:18)
at node.js:968:3
I'm guessing this is a Windows error. Anyone have any ideas?
Use {shell: true} in the options of spawn
I was hit with this problem recently so decided to add my findings here. I finally found the simplest solution in the Node.js documentation. It explains that:
child_process.exec() runs with shell
child_process.execFile() runs without shell
child_process.spawn() runs without shell (by default)
This is actually why the exec and spawn behave differently. So to get all the shell commands and any executable files available in spawn, like in your regular shell, it's enough to run:
const { spawn } = require('child_process')
const myChildProc = spawn('my-command', ['my', 'args'], {shell: true})
or to have a universal statement for different operating systems you can use
const myChildProc = spawn('my-command', ['my', 'args'], {shell: process.platform == 'win32'})
Side notes:
It migh make sense to use such a universal statement even if one primairly uses a non-Windows system in order to achieve full interoperability
For full consistence of the Node.js child_process commands it would be helpful to have spawn (with shell) and spawnFile (without shell) to reflect exec and execFile and avoid this kind of confusions.
I got it. On Windows bunyan isn't recognized in the console as a program but as a command. So to invoke it the use of cmd was needed. I also had to install bunyan globally so that the console could access it.
if (!/^win/.test(process.platform)) { // linux
var sp = spawn('bunyan', ['-o', 'short'], {
stdio: [null, process.stdout, process.stderr]
});
} else { // windows
var sp = spawn('cmd', ['/s', '/c', 'bunyan', '-o', 'short'], {
stdio: [null, process.stdout, process.stderr]
});
}
I solved same problem using cross-spawn. It allows me to spawn command on both windows and mac os as one common command.
I think you'll find that it simply can't find 'bunyun', but if you appended '.exe' it would work. Without using the shell, it is looking for an exact filename match to run the file itself.
When you use the shell option, it goes through matching executable extensions and finds a match that way. So, you can save some overhead by just appended the executable extension of your binary.
I was having this same problem when trying to execute a program in the current working directory in Windows. I solved it by passing the options { shell: true, cwd: __dirname } in the spawn() call. Then everything worked, with every argument passed as an array (not attached to the program name being run).
I think, the path of bin or something could be wrong. ENOENT = [E]rror [NO] [ENT]ry
I need to use node to generate diffs of some files. I have tried the diff npm package, and while that works great it is much less performant than the version of diff you will find in /usr/bin/.
When I try to use exec to run the diff command it always errors.
var childProcess = require('child_process');
var cmd = "diff /path/to/file1.txt /path/to/file2.txt";
childProcess.exec(cmd, (error, stdout, stderr) => {
if(error) {
console.log(error);
} else {
console.log(stdout);
}
});
The output:
{ [Error: Command failed: /bin/sh -c diff /path/to/file1.txt /path/to/file2.txt
]
killed: false,
code: 1,
signal: null,
cmd: '/bin/sh -c diff /path/to/file1.txt /path/to/file2.txt' }
If I run the command on the command line myself it works fine.
I've tried running /usr/bin/diff instead of just diff.
I've tried various different forms of quoting things.
Every other command I have tried, using the exact same files, has worked. cat, wc, etc.
Any thoughts?
Welp, I am answering my own question.
The diff command returns a failure exit code if it finds a difference. Therefore the console.log(stdout) call was never being reached.
If I ignore the error, everything works.
var childProcess = require('child_process');
var cmd = "diff /path/to/file1.txt /path/to/file2.txt";
childProcess.exec(cmd, (error, stdout, stderr) => {
console.log(stdout);
});
works like a charm.