SSH - npm seems to break ssh non-pty shells - node.js

I am developing a library/tool which lets the user execute arbitrary commands via SSH. Overall, this lib/tool will serve as software deployment tool, which needs access to remote machines to execute several commands (cd, mkdir, git ..., npm install, scp, etc).
While it basically works to execute remote commands via SSH, it seems that every time the command npm install is executed, the SSH connection gets terminated. I cannot tell what is causing this, but this very simple Node.js script can demonstrate it:
const spawn = require('child_process').spawn;
const bat = spawn('ssh', ['-T', '-oRequestTTY=no', '-oBatchMode=yes', 'user#host']);
bat.stdout.on('data', (data) => console.log('STDOUT: ' + data.toString()));
bat.stderr.on('data', (data) => console.log('STDERR: ' + data.toString()));
bat.on('exit', (code) => console.log(`Child exited with code ${code}`));
setTimeout(() => {
bat.stdin.write('pwd -P\n');
bat.stdin.write('cd someDir\n');
bat.stdin.write('npm install\n');
setTimeout(() => bat.stdin.write('pwd -P\n'), 2000);
}, 2000);
This will break/terminate the forked SSH process after npm install, so the delayed pwd -P will also fail. Removing the npm install command will make the SSH process stay intact until the app is terminated by the user.
I have actually faced this problem when I was working with the c library libssh, which had the very same issue, although I failed to notice that the npm install command seems to actually trigger the problem. See this related post: Channel in libssh gets closed for no obvious reason
What I found out is:
1. I am using a non-pty SSH shell
2. The libsshpacket-level debug output shows a packet 98 sent by the server just before the connection is closed
3. According to the RFC, packet 98 is SSH_MSG_CHANNEL_REQUEST, which can also be used to request a pty
So my asumption is, that I am working on a non-pty shell over SSH, and that something in the npm programm directly or indirectly leads to a server-side request for a pty, which cannot be handled by the non-pty shell I am working on, and thus, the SSH connection is closed on protocol-level.
Now my question is, what could be causing this, and is there a way to get rid of this problem?
Update may 29th
Actually, I was able to investigate this issue further. I started editing npm-cli.js and commented out everything, then step-by-step uncommented the lines, to see what is going to trigger the above behaviour.
At first, it seemed like the set-blocking module included via npmlog was causing the issue, but after also commenting out the actual code of set-blocking in index.js:4 (which is only stream._handle.setBlocking(blocking)), the bad behaviour still occurs, which confused the fuck out of me. Experimenting further revealed that actually the single piece of process.stdout is causing the whole issue.
To verify, I did the following:
Comment-out lines 22-94 in npm-cli.js. Executing npm install now will essentially do nothing. Also, running my example program above will not have the error.
Add the following code to npm-cli.js:19: if (process.stdout) ;. This also essentially does nothing, but it leads to the error occur again if executing the above test program.
Do the opposite test and change if (process.stdout) ; to if (process) ; - the test program will now run again without error.
So basically the above Node.js testprogram which demonstrates the error can be changed so that instead of npm install a simple oneline script is invoked, which results in the same error:
const spawn = require('child_process').spawn;
const bat = spawn('ssh', ['user#host']);
setTimeout(() => {
bat.stdin.write('node test.js\n'); // this breaks it
setTimeout(() => bat.stdin.write('pwd -P\n'), 2000);
}, 2000);
With test.js containing:
if (process.stdout) ; // only this line
I dont know how the process global object is constructed, or what is going on here, but something in Node.js' process/process.stdout object is somehow breaking SSH connections. I therefore think this is not an actual issue of npm, only indirectly.
Could anyone help to clarify?

Related

Execution via node js script with problems

Good morning, I have a problem running a command on the linux terminal and returning it to an api, for some reason I get a response on the server that has nothing to do with it, it's as if I just ran the "show route". The answer is in an array but this is normal, the problem is that they have nothing to do with the manual command.
Code:
Manual command:
Result the server receives:
Try
exec('/usr/sbin/birdc show route', (err, stdout, stderr)=>{})

mocha-phantomjs-core - slimerjs hangs without any error

Using mocha-phantomjs-core with slimerjs
I manage to run my tests successfully from CMD:
slimerjs mocha-phantomjs-core.js tests.html tap
Slimerjs window opens, I see the a browser window and all seems good, but the CMD doesn't finish (seems to wait for something). nothing is happening until I close the slimerjs window. I want to output the test result (using TAP reporter) as a file.
is that possible?
https://github.com/nathanboktae/mocha-phantomjs-core/issues/25
system.stderr.writeLine doesn't work on CMD or GIT bash... I've changed mocha-phantomjs-core.js fail function stderr to do stdout instead. now I get the error:
Likely due to external resource loading and timing, your tests require
calling window.initMochaPhantomJS() before calling any mocha setup
functions. See #12
So I had to add window.initMochaPhantomJS() before the setup function.. how silly! all this because I couldn't see any error due to the stderr issue not printed

Gulp.js process working on dev by not test/prod

I have a gulp.js process using the gulp-phantom plugin that works perfectly on my dev setup, Mac OS X 10.10, however on my test / prod environment (EC2 Amazon Linux) it just doesn't work at all, however it also isn't giving any sort of error message or any other helpful output, the task just starts and finishes again almost straight away:
Dev environment output:
$ gulp crawlSite
[17:39:19] Using gulpfile ~/Documents/dev/mysite.co.uk/gulpfile.js
[17:39:19] Starting 'crawlSite'...
[17:40:15] Finished 'crawlSite' after 57 s
Test environment output:
$ gulp crawlSite
[17:34:27] Using gulpfile /var/www/html/mysite.co.uk/gulpfile.js
[17:34:27] Starting 'crawlSite'...
[17:34:27] Finished 'crawlSite' after 715 ms
As you can see on the dev environment the process takes 57 seconds however on test it is only 715 milliseconds and on test it is not creating the files that my phantom script should be creating. My gulp task is very simple:
gulp.task('crawlSite', function() {
return gulp.src("phantom-crawl-website.js")
.pipe(phantom());
});
and my phantom script "phantom-crawl-website.js" file is in the same directory as the gulpfile.js file.
I have check that all the node modules are installed and that PhantomJS is installed globally on the test environment and everything checks out ok. If I run:
$ phantomjs phantom-crawl-website.js
from the command prompt on the test environment that works fine and it crawls the site and creates the files.
I have tried to use the gulp-phantom options for "debug" however I can never seem to see any output from this. I have tried using gulp-debug as well as follows:
gulp.task('crawlSite', function() {
return gulp.src("phantom-crawl-website.js")
.pipe(phantom({debug: true}))
.pipe(debug());
});
However all this does is give me the gulp-phantom output filename ("phantom-crawl-website.txt"). I have also tried to write the gulp-phantom output file in the following way:
gulp.task('crawlSite', function() {
return gulp.src("phantom-crawl-website.js")
.pipe(phantom({debug:true}))
.pipe(gulp.dest("./phantomOutput/"));
});
But all I get from this is a blank file created in the "phantomOutput" directory called "phantom-crawl-website.txt".
Can anyone advise what I am doing wrong and how I would be able to see the phantomJS debug output so I can work out what the problem is.
Thanks so much in advance.
UPDATE
I've managed to get some output from the gulp-phantom process by adding the following to the gulp-phantom index.js file:
program.stderr.on('data', function (data) {
console.log('stderr: ' + data);
});
Once this was added I'm now getting the following error message:
stderr: Can't open '/dev/stdin'
But still no luck actually getting it to work.
Found the issue. In the gulp-phantom module there appears to be an error with it using /dev/stdin were phantomjs expecting the phantom filename to be passed. On Mac OS X the /dev/stdin contains the contents of the file but on Linux it is denied permission to read it.
To fix it I removed the line that was pushing '/dev/stdin' into the arguments stack and then added one a bit further down in the "through" function call to pass the full path and filename to the phantomjs process instead.
I will issue a pull request to the gulp-phantom module creator and see if they accept this as fix for the issue.

Buildstops not creating file with exec in node js

I'm running two commands
indexer idx_name --rotate
indexer idx_name --buildstops dict_file 10
Everything is fine when I run these commands from command line. However, when I pass these two commands through my node application using exec, first command works successfully and for second command the dict_file is not getting generated.
I tried some combinations with sudo, but it didn't help. I checked the stdout from both these ways(node and shell) and it looked same.
Here is my node js code:
var exec = require('child_process').exec;
var cmd = 'indexer idx_name --rotate && indexer idx_name --buildstops dict_file 10';
exec(cmd, function(err, stdout, stderr) {
console.log(stdout);
});
Is there something I'm missing ?
Which ever user is running node, will need permission to write dict_file.
Might even find it easier to delete the file, and let it be created by the right user via node (assuming that user can write the the folder)
Sudo could also work, but will need to make sure the user running node, has sudo permissions. Sorting that is definitly outside the remit of stackoverflow.
... do also check you looking in the right place. In your example you dont show a path, so the file dict_file will just be created in the current working directory (not sure how node configures that)

How to execute shell script from hubot

I got my first hubot up and running, and wrote my first few scripts based on the existing examples. My existing workflow, which I would like to integrate with hubot, is essentially based on several shell scripts, each one of them performing one task. The task can be relatively complex (git/svn checkout, compiling code with gcc, and running it). How can I execute a bash script with hubot? I have seen this question, but it only addresses simple commands such as ls. I tried
build = spawn 'source', ['test.sh']
build.stdout.on 'data', (data) -> msg.send data.toString()
build.stderr.on 'data', (data) -> msg.send data.toString()
without any luck:
Hubot> execvp(): Permission denied
I checked the obvious things (-rwxr-xr-x permissions), and export HUBOT_LOG_LEVEL="debug".
I am running hubot with the same user that owns the bash scripts.
Thanks.
For reference: the answer was
build = spawn '/bin/bash', ['test.sh']
Dah
npm install hubot-script-shellcmd
is your doorway to the shell.

Resources