Need to insert variable value into sh step that using sed, but receive empty result
Here is the code:
#!/usr/bin/env groovy
def call(def cap_server){
dir("ruby"){
cap_server="pidor"
sh '''bundle install'''
sh '''sed -i "1i server '${cap_server}', user: 'ubuntu', roles: %w{web app db sidekiq}, ssh_options:{ keys: %w(~/.ssh/id_rsa)}" config/deploy/qa.rb'''
sh '''bundle exec cap qa deploy'''
}
}
And here is the result:
+ sed -i 1i server '', user: 'ubuntu', roles: %w{web app db sidekiq}, ssh_options:{ keys: %w(~/.ssh/id_rsa)} config/deploy/qa.rb
You need to use double quotes to create a strong template
sh """sed -i "1i server '${cap_server}', user: 'ubuntu', roles: %w{web app db sidekiq}, ssh_options:{ keys: %w(~/.ssh/id_rsa)}" config/deploy/qa.rb"""
Related
I have a piece of code that runs like this
package core.jenkins
class Utils implements Serializable {
def script
Utils(script) {
this.script = script
}
def func() {
script.withCredentials([script.usernamePassword(credentialsId: 'chartmuseum-basic-auth', usernameVariable: 'USER', passwordVariable: 'PASSWORD')]) {
script.sh "helm repo add --username script.USER} --password ${script.PASSWORD} chartmuseum \"http://${chartmuseumHostname}:8080\""
}
}
The above works perfectly fine but I do not a warning
Warning: A secret was passed to "sh" using Groovy String interpolation, which is insecure.
Affected argument(s) used the following variable(s): [PASSWORD, USER]
See https://jenkins.io/redirect/groovy-string-interpolation for details.
+ helm repo add --username **** --password **** chartmuseum http://apps-chartmuseum.apps.svc.cluster.local:8080
So following the guide, Im doing the following
script.withCredentials([script.usernamePassword(credentialsId: 'chartmuseum-basic-auth', usernameVariable: 'USER', passwordVariable: 'PASSWORD')]) {
script.sh 'helm repo add --username $script.USER --password $script.PASSWORD chartmuseum "http://$chartmuseumHostname:8080"'
}
But running the variable values are not be properly substitured and I get
+ helm repo add --username .USER --password .PASSWORD chartmuseum http://:8080
Error: Looks like "http://:8080" is not a valid chart repository or cannot be reached: Get http://:8080/index.yaml: dial tcp :8080: connect: connection refused
So neither the credentials nor the value of the chartmuseumHostname variable is being substituted correctly. What am I missing here ?
Actuall withCredentials() creates a environment variable which you can access it from shell scripts.
See here: https://www.jenkins.io/doc/pipeline/steps/credentials-binding/
Try using directly the shell variables:
script.sh 'helm repo add --username $USER --password $PASSWORD chartmuseum "http://$chartmuseumHostname:8080"'
Just binding together the answers already on this post, the withCredentials makes it so that you should be able to use the variables directly (answer by #catalin), the single quotes make it so that jenkins should stop complaining about security and if you want to be extra careful, you can double quote the variable values as suggested in the docs for withCredentials.
This should give you something like this:
script.withCredentials([script.usernamePassword(credentialsId: 'chartmuseum-basic-auth',
usernameVariable: 'USER',
passwordVariable: 'PASSWORD')]) {
script.sh 'helm repo add --username "$USER" --password "$PASSWORD" chartmuseum "http://$chartmuseumHostname:8080"'
}
which still leaves us with the question of why you are calling things with the script. prefix as mentioned in the comments by #matt-schuchard.
I try using the suggestion by #Catalin (https://www.jenkins.io/doc/pipeline/steps/credentials-binding/ using directly the shell variables)
But for me adding double quotes inside single quotes doesn't work.
The only solution I found is taking the variables out of the single quotes like:
'myscript $secretvariable' + notsecretvariable
Examples:
Test1: Try using recommended solution (jenkins/#catalin)
Code:
sh label: 'Test1', script: 'echo this is a secret $docker_pwd this is not "$dockerRegistry"'
Result: variable dockerRegistry is not interpolated/resolved
15:50:43 [Pipeline] sh (Test1)
15:50:43 + echo this is a secret **** this is not ''
15:50:43 this is a secret **** this is not
Test2: Take non-sensitive variable out of the single quotes:
sh label: 'Test2', script: 'echo this is a secret $docker_pwd this is not' + dockerRegistry
Result: variable dockerRegistry is properlly resolved
15:50:44 [Pipeline] sh (Test2)
15:50:44 + echo this is a secret **** this is not my.repositories.xx
15:50:44 this is a secret **** this is not my.repositories.xx
I have NestJS application that uses TypeORM to connect to my local database. I create database with shell script:
#!/bin/bash
set -e
SERVER="my_database_server";
PW="mysecretpassword";
DB="my_database";
echo "echo stop & remove old docker [$SERVER] and starting new fresh instance of [$SERVER]"
(docker kill $SERVER || :) && \
(docker rm $SERVER || :) && \
docker run --name $SERVER -e POSTGRES_PASSWORD=$PW \
-e PGPASSWORD=$PW \
-p 5432:5432 \
-d postgres
# wait for pg to start
echo "sleep wait for pg-server [$SERVER] to start";
SLEEP 3;
# create the db
echo "CREATE DATABASE $DB ENCODING 'UTF-8';" | docker exec -i $SERVER psql -U postgres
echo "\l" | docker exec -i $SERVER psql -U postgres
After that, it logs databases:
Then I fire up my application, and I encounter error "error: database "my_database" does not exist"
I use following code to connect to database:
static getDatabaseConnection(): TypeOrmModuleOptions {
console.log(require('dotenv').config())
return {
type: 'postgres',
host: "127.0.0.1",
port: 5432,
username: 'postgres',
password: 'mysecretpassword',
database: 'my_database',
entities: ['dist/**/*.entity{.ts,.js}'],
synchronize: true,
};
}
Any ideas where do I go wrong?
When connecting to a docker instance, you should usually use the service name. In this case I guess it is my_database_server as host parameter.
return {
type: 'postgres',
host: "my_database_server",
port: 5432,
username: 'postgres',
password: 'mysecretpassword',
database: 'my_database',
entities: ['dist/**/*.entity{.ts,.js}'],
synchronize: true,
};
"localhost" isn't address of your docker container. Which address uses docker you can look running command:
$ docker inspect {your_container_name}
for me is: 172.17.0.2
Try enable SSL, adding next configuration lines:
ssl: true,
extra: { ssl: { rejectUnauthorized: false } }
Try using localhost instead of 127.0.0.1
I want to replace some variables in a file having $variablename, at runtime from jenkins pipeline script. It seems envsubst is the best for my use case. When i execute by command line on linux server its working fine but when i'm executing through jenkins pipeline in sh script, nothing happens.
sonar-scanner.properties:
sonar.projectKey=Project:MavenTest$BRANCHNAME
sonar.projectName=MavenTest$BRANCHNAME
Example of Command line on linux box:
$ export BRANCHNAME=develop
$ envsubst '$BRANCHNAME'
Output:
sonar.projectKey=Project:MavenTestdevelop
sonar.projectName=MavenTestdevelop
But when i'm executing through jenkins file as a script, nothing is changed in file.
jenkins script:
node {
stage('checkout'){
checkout([$class: 'GitSCM', branches: [[name: ':^(?!origin/master$|origin/develop$).*']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'c0ce73db-3864-4360-9c17-d87caf8a9ea5', url: 'http://172.16.4.158:17990/scm/ctoo/testmaven.git']]])
}
stage('initialize variables'){
// Configuring BRANCH_NAME variable
sh 'git name-rev --name-only HEAD > GIT_BRANCH'
sh label: '', script: 'cut -d \'/\' -f 3 GIT_BRANCH > BRANCH'
branchname = readFile('BRANCH').trim()
env.BRANCHNAME = branchname
}
stage('build & SonarQube analysis') {
withSonarQubeEnv('Sonar') {
sh "envsubst '$BRANCHNAME' <sonar-scanner.properties"
}
}
}
Output:
[Pipeline] sh (hide)
envsubst repotest
sonar.projectKey=Project:MavenTest$BRANCHNAME
sonar.projectName=MavenTest$BRANCHNAME
Can someone please help me
Hi I don't have idea about the envbust but this can be achieved by passing sonar parameters via command line to the sonar see the below example:
withSonarQubeEnv('Sonar') {
sh "<sonarscanner path> -Dsonar.projectKey=Project:MavenTest$BRANCHNAME"
}
I had this problem and solved it by using his escape character
\,
for example:
sh "envsubst '\${SERVER_NAME}' < ./config/nginx/nginx.conf.template > ./config/nginx/nginx.conf"
I created Rundeck Key storage and stored password in it
Then created Job option
Then in inline script i specified folowing (keys/JIRA is Rundeck password storage)
curl -XN -u user:keys/JIRA
But password is not passed and authnetication fails, what am i doing wrong ?
The password value will be expanded when it is passed to the script. Below is an example:
- description: ''
executionEnabled: true
id: 1f7f5312-0887-4841-a7ef-1c30f712f927
loglevel: INFO
name: How to pass Rundeck key storage to script
nodeFilterEditable: false
options:
- name: JiraPass
secure: true
storagePath: keys/jira.password
valueExposed: true
scheduleEnabled: true
sequence:
commands:
- args: ${option.JiraPass}
script: |
#!/usr/bin/env bash
jira_password=$1
echo curl -XN -u "user:$1"
keepgoing: false
strategy: node-first
uuid: 1f7f5312-0887-4841-a7ef-1c30f712f927
I am unable to execute a hive -e command using simple-ssh module .exec() in nodejs.
I think there is a problem with the single/double quotes ' or ". I don't know which quote to put in what sequence. I tried a lot of combination, but none of them worked.
Here is the code below:
var runSSH(obj){
var ssh = new SSH({
host: remote1,
user: 'root',
timeout: 1500000,
key: require('fs').readFileSync("C:/Users/Aiman/Desktop/hRep_prv"),
agent: process.env.SSH_AUTH_SOCK,
agentForward: true
});
ssh.exec('timeout 900 ssh -i /root/rsaPrvtKeyPath/to/remoteHost2 '+remoteHost2+' \'for i in '+remote3+' '+remote4+'; do clush -w ${i} "hive -e \'select * from table_name limit 3;\'" done\' ',{
out: function(stdout) {
devHive_check += stdout;
obj.devHive_check = devHive_check;
console.log(stdout);
}
}) //-->not executing
.exec('timeout 300 ssh -i /root/rsaPrvtKeyPath/to/remoteHost2 '+remoteHost2+' \'for i in '+remote3+' '+remote4+'; do clush -w ${i} "ps -ef | grep HiveServer2"; done;\' ',{
out: function(stdout) {
devHS2_check += stdout;
obj.devHS2_check = devHS2_check;
console.log(stdout);
}
})//-->running fine
.exec('echo "parse and save"',{
out: function(){
parseData(obj);
ssh.end();
}
}).start(); //-->running fine
}
I am logging into remoteHost1, running a couple of shell scripts (which are runnig fine), then I am doing an ssh to remoteHost2 to check Hive (hive is running on remote3 and remote4).
HiveServer2 is running fine, but Hive isn't.
Please help me.
Its solved now.
Instead of the single quotes inside hive, it requires a doulbe quotes, i.e. instead line ssh.exec('timeout 900 ssh -i /root/rsaPrvtKeyPath/to/remoteHost2 '+remoteHost2+' \'for i in '+remote3+' '+remote4+'; do clush -w ${i} "hive -e \'select * from table_name limit 3;\'" done\' ',{
I did
ssh.exec('timeout 900 ssh -i /root/rsaPrvtKeyPath/to/remoteHost2 '+remoteHost2+' \'for i in '+remote3+' '+remote4+'; do clush -w ${i} "hive -e \\\"select * from table_name limit 3;\\\" " done\' ',{, and it worked.
Thanks #mscdex for your support.