Unable to execute "hive -e 'select * from table" using .exec() method of simple-ssh npm module - node.js

I am unable to execute a hive -e command using simple-ssh module .exec() in nodejs.
I think there is a problem with the single/double quotes ' or ". I don't know which quote to put in what sequence. I tried a lot of combination, but none of them worked.
Here is the code below:
var runSSH(obj){
var ssh = new SSH({
host: remote1,
user: 'root',
timeout: 1500000,
key: require('fs').readFileSync("C:/Users/Aiman/Desktop/hRep_prv"),
agent: process.env.SSH_AUTH_SOCK,
agentForward: true
});
ssh.exec('timeout 900 ssh -i /root/rsaPrvtKeyPath/to/remoteHost2 '+remoteHost2+' \'for i in '+remote3+' '+remote4+'; do clush -w ${i} "hive -e \'select * from table_name limit 3;\'" done\' ',{
out: function(stdout) {
devHive_check += stdout;
obj.devHive_check = devHive_check;
console.log(stdout);
}
}) //-->not executing
.exec('timeout 300 ssh -i /root/rsaPrvtKeyPath/to/remoteHost2 '+remoteHost2+' \'for i in '+remote3+' '+remote4+'; do clush -w ${i} "ps -ef | grep HiveServer2"; done;\' ',{
out: function(stdout) {
devHS2_check += stdout;
obj.devHS2_check = devHS2_check;
console.log(stdout);
}
})//-->running fine
.exec('echo "parse and save"',{
out: function(){
parseData(obj);
ssh.end();
}
}).start(); //-->running fine
}
I am logging into remoteHost1, running a couple of shell scripts (which are runnig fine), then I am doing an ssh to remoteHost2 to check Hive (hive is running on remote3 and remote4).
HiveServer2 is running fine, but Hive isn't.
Please help me.

Its solved now.
Instead of the single quotes inside hive, it requires a doulbe quotes, i.e. instead line ssh.exec('timeout 900 ssh -i /root/rsaPrvtKeyPath/to/remoteHost2 '+remoteHost2+' \'for i in '+remote3+' '+remote4+'; do clush -w ${i} "hive -e \'select * from table_name limit 3;\'" done\' ',{
I did
ssh.exec('timeout 900 ssh -i /root/rsaPrvtKeyPath/to/remoteHost2 '+remoteHost2+' \'for i in '+remote3+' '+remote4+'; do clush -w ${i} "hive -e \\\"select * from table_name limit 3;\\\" " done\' ',{, and it worked.
Thanks #mscdex for your support.

Related

PowerShell script to SSH to Multiple Linux devices and restart/reboot them

I am comfortable with writing single queries. I am new to writing bash scripts trying to automate the daily stuff. I need help on creating a PowerShell or bash script where I can SSH to multiple Linux devices with same SSH key and then reboot the devices.
I access linux devices manually with the following command in PowerShell
ssh -i C:\<path-to-the-private-key\test.ppk test#XX.X.X.XXX (X - IP Address)
Then I enter the following command
sudo reboot
It asks me to type the password and then restarts the device.
I have 100+ devices that I need to restart.
I can get a list of all IP address in a text file. How can we search for all the IP address in a text file, authenticate with the SSH private key and run the sudo command to restart the device?
Would it also be possible to throw a message if it was not able to restart a device?
Any help would be appreciated.
This is the script that I have.
testreboot.sh
#!/bin/bash
pw="test123"
hosts='IP.txt'
while read -r line; do {
/usr/bin/expect << EOF do
ssh test#"$hosts" 'sudo reboot'
expect "*?assword*"
send "%pw\r"
EOF
}
done < $hosts
IP.txt
XXX.XX.XX.XX
XXX.XX.XX.XX
XXX.XX.XX.XX
XXX.XX.XX.XX
I have Ubuntu 20.04 installed from Windows App Store. I am trying to run the testreboot.sh from PowerShell using the following command and get the following error message.
bash testreboot.sh
testreboot.sh: line 2: $'\r': command not found
testreboot.sh: line 3: $'\r': command not found
testreboot.sh: line 5: $'\r': command not found
testreboot.sh: line 16: syntax error near unexpected token `done'
testreboot.sh: line 16: `done < $hosts'
A better solution to this problem is to use something like Ansible, Chef, or Puppet to solve these multi-server coordination needs.
Here is an example in a shell script using expect to ssh to another server and login:
NOTE: When you use expect in this manner, you need to escape " and $ and
other items. If password has $, it must be escaped.
This is where after logging in and expect see a command prompt
For example:
44 [localhost.localdomain]/home/map%
This is where you would need to add sudo reboot command
-re \"$reg_ex_prompt\" {
}
Test Script:
#!/bin/sh
#
debug=0
exit_val=0
spawn_item="ssh"
destination="user_name#<IP of server to shh>"
reg_ex_prompt='\[%|>|\$|#\] $'
#
# Change -D 0 to -D 1 to debug interactivly
#
expect -D $debug -c "
spawn $spawn_item $destination
set ret 1
set timeout 20
expect {
timeout {
send_user \"Timeout reached!\"
exit \$ret
}
eof {
puts \"End of test connection reached!\"
if { \$ret == 0 } {
puts \"Connection test Successful!\"
puts \"Exiting $destination ...\"
} else {
puts \"Connection Failure!\"
}
exit \$ret
}
\"Connection refused\" {
puts \"ERROR: Trouble connecting to $device_type_parm destination $destination\"
puts \"Aborting...\"
exit \$ret
}
\"Permission denied\" {
puts \"ERROR: User name or password is incorrect for $destination\"
puts \"Aborting...\"
exit \$ret
}
\"Connection reset by peer\" {
puts \"ERROR: Trouble connecting to $destination\"
puts \"Aborting...\"
exit \$ret
}
\"you sure you want to continue connecting\" {
send \"yes\r\"
exp_continue
}
\"assword:\" {
puts \"\r\nSending password\"
send \"password\\\$\r\"
exp_continue
}
\"Choice? \" {
send \"1\r\"
exp_continue
}
\"Q. Quit\" {
send \"q\r\"
exp_continue
}
-re \"$reg_ex_prompt\" {
send \"sudo reboot\r\"
sleep 2
set ret 0
exit \$ret
}
interact
} "
# get the exit value from expect statment above
exit_val=$?

BashScript stopps/ never finishes without any errormessage

when i try to run a bashscript in GIT BASH (working on a Windows OS) to execute psql commands i get the following result:
name#DESKTOP-AVB2MRC MINGW64 ~/dragonstack/backend
a npm run configure
> backend#1.0.0 configure C:\Users\name\dragonstack\backend
> bash ./bin/configure_db.sh
Configuring dragonstackdb
My Files:
configure_db.sh
#!/bin/bash
echo "Configuring dragonstackdb";
dropdb -U node_user dragonstackdb;
createdb -U node_user dragonstackdb;
psql -U node_user dragonstackdb <./bin/sql/generation.sql;
psql -U node_user dragonstackdb <./bin/sql/dragon.sql;
echo "dragonstackdb configured";
generation.sql
CREATE TABLE generation(
id SERIAL PRIMARY KEY ,
expiration TIMESTAMP NOT NULL
);
dragon.sql
CREATE TABLE dragon(
id SERIAL PRIMARY KEY,
birthdate TIMESTAMP NOT NULL,
nickname VARCHAR(64),
"generationId" INTEGER,
FOREIGN KEY ("generationId") REFERENCES generation(id)
);

How to access value of a jenkins groovy variable in shell script for loop

When i am passing value of a variable declared in jenkins Groovy script its value is not retained in for loop which is running on a remote server. Strange thing is i am able to access the same value outside the for loop.
Here is the sample code i am trying to use
#!/usr/bin/env groovy
def config
def COMMANDS_TO_CHECK='curl grep hello awk tr mkdir bc'
pipeline {
agent {
label "master"
}
stages {
stage ('Validation of commands') {
steps {
script {
sh """
#!/bin/bash
/usr/bin/sshpass -p passwrd ssh user#host << EOF
hostname
echo $COMMANDS_TO_CHECK ---> This is printed
for CURRENT_COMMAND in \$COMMANDS_TO_CHECK
do
echo ${CURRENT_COMMAND} ---> Why This is not printed?
echo \${CURRENT_COMMAND} ----> Why This is not printed?
done
hostname
EOF
exit
"""
}
}
}
}
}
Output
[workspace#3] Running shell script
+ /usr/bin/sshpass -p passwrd ssh user#host
Pseudo-terminal will not be allocated because stdin is not a terminal.
illinsduck01
curl grep hello awk tr mkdir bc
illinsduck01
+ exit
You can wrap sh in """ ... """ as below
#!/usr/bin/env groovy
def config
pipeline {
agent {
label "master"
}
stages {
stage ('Validation of commands') {
steps {
script {
sh """#!/bin/sh
/usr/bin/sshpass -p password ssh username#hostname << EOF
COMMANDS_TO_CHECK="curl grep hello awk tr mkdir bc"
hostname
echo \$COMMANDS_TO_CHECK
for CURRENT_COMMAND in \$COMMANDS_TO_CHECK
do
echo \$CURRENT_COMMAND
which \$CURRENT_COMMAND
status=\$?
if [ \${status} -eq 0 ]
then
echo \${CURRENT_COMMAND} command is OK
else
echo "Failed to find the \${CURRENT_COMMAND} command"
fi
done
hostname
EOF
exit
"""
}
}
}
}
}

escape quotes using sed in jenkins pipline sh step

Need to insert variable value into sh step that using sed, but receive empty result
Here is the code:
#!/usr/bin/env groovy
def call(def cap_server){
dir("ruby"){
cap_server="pidor"
sh '''bundle install'''
sh '''sed -i "1i server '${cap_server}', user: 'ubuntu', roles: %w{web app db sidekiq}, ssh_options:{ keys: %w(~/.ssh/id_rsa)}" config/deploy/qa.rb'''
sh '''bundle exec cap qa deploy'''
}
}
And here is the result:
+ sed -i 1i server '', user: 'ubuntu', roles: %w{web app db sidekiq}, ssh_options:{ keys: %w(~/.ssh/id_rsa)} config/deploy/qa.rb
You need to use double quotes to create a strong template
sh """sed -i "1i server '${cap_server}', user: 'ubuntu', roles: %w{web app db sidekiq}, ssh_options:{ keys: %w(~/.ssh/id_rsa)}" config/deploy/qa.rb"""

Downloading Java JDK through Script

I have been using the following to download JDK 8u112 via a script.
wget --no-check-certificate --no-cookies --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/8u112-b15/jdk-8u112-linux-x64.tar.gz
Recently, it throws ERROR 404: Not found and when you go to the link, it shows the same text in Downloading Java JDK on Linux via wget is shown license page instead
I've also tried http://download.oracle.com/otn/java/jdk/8u112-b15/jdk-8u112-linux-x64.tar.gz but it throws 401 Authorization Error.
Is there a new work around on this?
It seems like the most recent version of jdk can be downloaded by wget but not the files in the archives.
As such, I'm using casper.js script to login to Oracle and to download.
Following is my script to download Japanese version of jdk8u121. The current script will only attempt to download but will fail on redirect. I'm using download.sh bash script to scan the log to get the url with session parameter and using wget to do the actual download.
You'll need to replace <username> and <password> with valid ones to login to Oracle site.
Change values of jdkTag and jdkFileLink to get the jdk version you want to download.
oraclejdk.js
var casper = require('casper').create({
verbose: true,
logLevel: 'info', // debug
pageSettings: {
userAgent: "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36",
loadImages: false,
loadPlugins: false
}
});
// login info
var loginUrl='http://www.oracle.com/webapps/redirect/signon?nexturl=https://www.oracle.com/technetwork/java/javase/downloads/java-archive-javase8-2177648.html';
var username='<username>';
var password='<password>';
// accept license page info
var jdkUrl='http://www.oracle.com/technetwork/';
var jdkTag='jdk-8u121-oth-JPR';
// download jdk info
var jdkFileLink='jdk-8u121-oth-JPRXXXjdk-8u121-linux-x64.tar.gz';
// open login page
casper.start(loginUrl);
casper.thenEvaluate(function(username, password) {
// this.capture('loginPage.png', {top:0, left:0, width:600, height:800});
document.querySelector("#sso_username").value = username;
document.querySelector("#ssopassword").value = password;
doLogin(document.LoginForm);
}, {
username: username,
password: password
});
// login to oracle site
casper.then(function() {
this.waitForUrl(jdkUrl, function() {
// this.capture('jdkPage.png', {top:0, left:0, width:1200, height:800});
this.evaluate(function(jdkTag) {
disableDownloadAnchors(document, false, jdkTag);
hideAgreementDiv(document, jdkTag);
writeSessionCookie('oraclelicense', 'accept-securebackup-cookie');
}, {
jdkTag: jdkTag
});
}, null, null, 5000);
});
// accept license
casper.then(function() {
this.waitFor(function checkLink() {
return this.evaluate(function(jdkTag) {
return (document.getElementById('agreementDiv' + jdkTag).getAttribute('style') === 'visibility: hidden;');
}, {
jdkTag: jdkTag
});
}, function then() {
// this.capture('acceptedLicense.png', {top:0, left:0, width:1200, height:800});
downlink = this.evaluate(function(jdkFileLink) {
var jdkElement = document.getElementById(jdkFileLink);
if (jdkElement) {
var jdkLink = jdkElement.getAttribute("href");
jdkElement.click();
return jdkLink;
}
}, {
jdkFileLink: jdkFileLink
});
}, null, 5000);
});
casper.run();
download.sh
#!/bin/bash
url=$(casperjs --web-security=no oraclejdk.js |grep "http://download.oracle.com/otn/java/jdk" $() | sed -e 's/^.*: //')
jdk=$(echo "${url}" | sed -e 's/^.*jdk-/jdk/' |sed -e 's/?.*//')
wget -O "${jdk}" "${url}"
This is not a direct answer to your question...but Here is how i get URL to latest jdk download URL
#!/bin/bash
jdkwebinstallerDownloadPage="https://www.oracle.com"$(curl -s https://www.oracle.com/technetwork/java/javase/downloads/index.html | unix2dos | grep "<a name=\"JDK8\"" | sed 's/^.*\<a name=\"JDK8\" href=//g' | sed -r 's/>.*//g' | sed s/\"//g)
## Above yields https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
jdkinstallerDownloadURL=$(curl -s $jdkwebinstallerDownloadPage | grep windows | grep downloads | grep x64 | grep jdk | grep -v demos | sed -r 's/^.*https/https/g' | sed -r 's/\".*//g')
## yields https://download.oracle.com/otn/java/jdk/8u221-b11/230deb18db3e4014bb8e3e8324f81b43/jdk-8u221-windows-x64.exe
I am now looking to how to download from this url using wget...given that i have cedentials to login into oracle's login webpage which is https://login.oracle.com/mysso/signon.jsp

Resources