Unable to interpolate sensitive environment variables - groovy

I have a piece of code that runs like this
package core.jenkins
class Utils implements Serializable {
def script
Utils(script) {
this.script = script
}
def func() {
script.withCredentials([script.usernamePassword(credentialsId: 'chartmuseum-basic-auth', usernameVariable: 'USER', passwordVariable: 'PASSWORD')]) {
script.sh "helm repo add --username script.USER} --password ${script.PASSWORD} chartmuseum \"http://${chartmuseumHostname}:8080\""
}
}
The above works perfectly fine but I do not a warning
Warning: A secret was passed to "sh" using Groovy String interpolation, which is insecure.
Affected argument(s) used the following variable(s): [PASSWORD, USER]
See https://jenkins.io/redirect/groovy-string-interpolation for details.
+ helm repo add --username **** --password **** chartmuseum http://apps-chartmuseum.apps.svc.cluster.local:8080
So following the guide, Im doing the following
script.withCredentials([script.usernamePassword(credentialsId: 'chartmuseum-basic-auth', usernameVariable: 'USER', passwordVariable: 'PASSWORD')]) {
script.sh 'helm repo add --username $script.USER --password $script.PASSWORD chartmuseum "http://$chartmuseumHostname:8080"'
}
But running the variable values are not be properly substitured and I get
+ helm repo add --username .USER --password .PASSWORD chartmuseum http://:8080
Error: Looks like "http://:8080" is not a valid chart repository or cannot be reached: Get http://:8080/index.yaml: dial tcp :8080: connect: connection refused
So neither the credentials nor the value of the chartmuseumHostname variable is being substituted correctly. What am I missing here ?

Actuall withCredentials() creates a environment variable which you can access it from shell scripts.
See here: https://www.jenkins.io/doc/pipeline/steps/credentials-binding/
Try using directly the shell variables:
script.sh 'helm repo add --username $USER --password $PASSWORD chartmuseum "http://$chartmuseumHostname:8080"'

Just binding together the answers already on this post, the withCredentials makes it so that you should be able to use the variables directly (answer by #catalin), the single quotes make it so that jenkins should stop complaining about security and if you want to be extra careful, you can double quote the variable values as suggested in the docs for withCredentials.
This should give you something like this:
script.withCredentials([script.usernamePassword(credentialsId: 'chartmuseum-basic-auth',
usernameVariable: 'USER',
passwordVariable: 'PASSWORD')]) {
script.sh 'helm repo add --username "$USER" --password "$PASSWORD" chartmuseum "http://$chartmuseumHostname:8080"'
}
which still leaves us with the question of why you are calling things with the script. prefix as mentioned in the comments by #matt-schuchard.

I try using the suggestion by #Catalin (https://www.jenkins.io/doc/pipeline/steps/credentials-binding/ using directly the shell variables)
But for me adding double quotes inside single quotes doesn't work.
The only solution I found is taking the variables out of the single quotes like:
'myscript $secretvariable' + notsecretvariable
Examples:
Test1: Try using recommended solution (jenkins/#catalin)
Code:
sh label: 'Test1', script: 'echo this is a secret $docker_pwd this is not "$dockerRegistry"'
Result: variable dockerRegistry is not interpolated/resolved
15:50:43 [Pipeline] sh (Test1)
15:50:43 + echo this is a secret **** this is not ''
15:50:43 this is a secret **** this is not
Test2: Take non-sensitive variable out of the single quotes:
sh label: 'Test2', script: 'echo this is a secret $docker_pwd this is not' + dockerRegistry
Result: variable dockerRegistry is properlly resolved
15:50:44 [Pipeline] sh (Test2)
15:50:44 + echo this is a secret **** this is not my.repositories.xx
15:50:44 this is a secret **** this is not my.repositories.xx

Related

Uploading a file into Nexus from Jenkins pipeline is not working

This doesn't work:
withCredentials([usernamePassword(credentialsId: 'xxxxxxxxxxx', passwordVariable: 'ABCD', usernameVariable: 'XYZ')]) {
dir("build") {
sh "curl -u $XYZ:$ABCD --upload-file xyz.tar.gz https://mnpqr/repository/abc/$BRANCH_DIR/xyz.tar.gz"
}
}
Error:
/var/lib/jenkins/workspace/xyz/build#tmp/durable-71c2368a/script.sh: line 1: VG7cJ: No such file or directory
But this works:
withCredentials([usernamePassword(credentialsId: 'xxxxxxxxxxx', passwordVariable: 'ABCD', usernameVariable: 'XYZ')]) {
dir("build") {
sh 'curl -u $XYZ:$ABCD --upload-file xyz.tar.gz https://mnpqr/repository/abc/$BRANCH_DIR/xyz.tar.gz'
}
}
I want to interpolate more data into the sh script but I cannot as its failing with double quotes.
You have to be careful to distinguish the variable scopes:
environment variables set in your script
environment variables set by Jenkins processes
local variables not available in forked shell process
They all have to be handled in a different way when replacing them in your double-quoted string:
node {
stage('My stage') {
// local variable scope
varContent = "Variable content"
// environment variable set up in script
env.envContent = "Environment content"
// environment variable available in shell script
withEnv(["BRANCH_NAME=myBranch"]) {
sh("echo ${varContent} XX ${env.envContent} XX \${envContent} XX \${BRANCH_NAME} XX ${env.BRANCH_NAME}")
}
}
}
In the example you see all of the three types. Let's have a closer look at the shell command:
sh("echo ${varContent} XX ${env.envContent} XX \${envContent} XX \${BRANCH_NAME} XX ${env.BRANCH_NAME}")
${varContent} is a variable from local script scope it is replaced before the string is written to a temporary shell script
${env.envContent} and ${env.BRANCH_NAME} handle environment variables that have already been set beforehand, as if they were a "local scope" variable
\${envContent} and \${BRANCH_NAME} are the actual environment variables. The backslash escapes the dollar sign, and the shell script will contain shell variable placeholders ${envContent} and ${BRANCH_NAME} that will be replaced at shell script run time.
Running the above script will show the following output:

Unable to test BaselinePipelineTest withCredentialInterceptor

I'm trying to test a function in my Jenkins Script Pipeline.
void doSomeDockerThings(){
withCredentials([[$class: 'UsernamePasswordMultiBinding', creditialsId: 'my_creds', usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD']]) {
sh "docker login -u $USERNAME -p '$PASSWORD' $DTR"
}
}
From a Groovy Unit Test that extends BasePipelineTest, I have the following test method.
#Test
void testDoSomeDockerThings(){
helper.registerAllowedMethod( "withCredentials", [LinkedHasMap.class, Closure.class], null );
helper.registerAllowedMethod( "sh", [String.class], {c -> return c; })
binding.setVariable( "USERNAME", "user" );
binding.setVariable( "PASSWORD", "password" );
binding.setVariable( "DTR", "Docker_Trusted_Registry" );
}
What I want to do is now write an assert that this command executed with what I expected. Since that is in a Closer of withCredentials how can I verify this?
sh "docker login -u $USERNAME -p '$PASSWORD' $DTR"
I've also tried using the withCredentialsInterceptor and I can't seem to get the syntax right for what it expects. If anyone has an example of that I would appreciate it.
helper.registerAllowedMethod( "withCredentials", [LinkedHasMap.class, Closure.class], withCredentialsInterceptor );
I'm really struggling to find documentation or examples are how to do this.

escape quotes using sed in jenkins pipline sh step

Need to insert variable value into sh step that using sed, but receive empty result
Here is the code:
#!/usr/bin/env groovy
def call(def cap_server){
dir("ruby"){
cap_server="pidor"
sh '''bundle install'''
sh '''sed -i "1i server '${cap_server}', user: 'ubuntu', roles: %w{web app db sidekiq}, ssh_options:{ keys: %w(~/.ssh/id_rsa)}" config/deploy/qa.rb'''
sh '''bundle exec cap qa deploy'''
}
}
And here is the result:
+ sed -i 1i server '', user: 'ubuntu', roles: %w{web app db sidekiq}, ssh_options:{ keys: %w(~/.ssh/id_rsa)} config/deploy/qa.rb
You need to use double quotes to create a strong template
sh """sed -i "1i server '${cap_server}', user: 'ubuntu', roles: %w{web app db sidekiq}, ssh_options:{ keys: %w(~/.ssh/id_rsa)}" config/deploy/qa.rb"""

replacing file variables by envsubst in jenkins pipeline

I want to replace some variables in a file having $variablename, at runtime from jenkins pipeline script. It seems envsubst is the best for my use case. When i execute by command line on linux server its working fine but when i'm executing through jenkins pipeline in sh script, nothing happens.
sonar-scanner.properties:
sonar.projectKey=Project:MavenTest$BRANCHNAME
sonar.projectName=MavenTest$BRANCHNAME
Example of Command line on linux box:
$ export BRANCHNAME=develop
$ envsubst '$BRANCHNAME'
Output:
sonar.projectKey=Project:MavenTestdevelop
sonar.projectName=MavenTestdevelop
But when i'm executing through jenkins file as a script, nothing is changed in file.
jenkins script:
node {
stage('checkout'){
checkout([$class: 'GitSCM', branches: [[name: ':^(?!origin/master$|origin/develop$).*']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'c0ce73db-3864-4360-9c17-d87caf8a9ea5', url: 'http://172.16.4.158:17990/scm/ctoo/testmaven.git']]])
}
stage('initialize variables'){
// Configuring BRANCH_NAME variable
sh 'git name-rev --name-only HEAD > GIT_BRANCH'
sh label: '', script: 'cut -d \'/\' -f 3 GIT_BRANCH > BRANCH'
branchname = readFile('BRANCH').trim()
env.BRANCHNAME = branchname
}
stage('build & SonarQube analysis') {
withSonarQubeEnv('Sonar') {
sh "envsubst '$BRANCHNAME' <sonar-scanner.properties"
}
}
}
Output:
[Pipeline] sh (hide)
envsubst repotest
sonar.projectKey=Project:MavenTest$BRANCHNAME
sonar.projectName=MavenTest$BRANCHNAME
Can someone please help me
Hi I don't have idea about the envbust but this can be achieved by passing sonar parameters via command line to the sonar see the below example:
withSonarQubeEnv('Sonar') {
sh "<sonarscanner path> -Dsonar.projectKey=Project:MavenTest$BRANCHNAME"
}
I had this problem and solved it by using his escape character
\,
for example:
sh "envsubst '\${SERVER_NAME}' < ./config/nginx/nginx.conf.template > ./config/nginx/nginx.conf"

How to pass Rundeck key storage to script

I created Rundeck Key storage and stored password in it
Then created Job option
Then in inline script i specified folowing (keys/JIRA is Rundeck password storage)
curl -XN -u user:keys/JIRA
But password is not passed and authnetication fails, what am i doing wrong ?
The password value will be expanded when it is passed to the script. Below is an example:
- description: ''
executionEnabled: true
id: 1f7f5312-0887-4841-a7ef-1c30f712f927
loglevel: INFO
name: How to pass Rundeck key storage to script
nodeFilterEditable: false
options:
- name: JiraPass
secure: true
storagePath: keys/jira.password
valueExposed: true
scheduleEnabled: true
sequence:
commands:
- args: ${option.JiraPass}
script: |
#!/usr/bin/env bash
jira_password=$1
echo curl -XN -u "user:$1"
keepgoing: false
strategy: node-first
uuid: 1f7f5312-0887-4841-a7ef-1c30f712f927

Resources