I have two scripts first on file system,second into jenkins job.
Second script calling the first and passed parameters into it.
Parameters contains password parameter.
How can I hide password into logs?
I have tried to hide output by using exec command but problem wasn't solved.
The Mask Passwords plugin does just that.
Please find below my findings with solution [without using Mask Passwords plugin]:
Brief Description about my jenkins job:
I wrote a job which downloads the artifacts from Nexus based on the parameters given at run-time and then makes a Database SQL connection and deploy the SQL scripts using maven flyway plugin. My job takes - Environment, Database Schema, Artifact version number, Flyway command, Database User and it's password as input parameters.
Brief Background about problem:
While passing the PASSWORD as MAVEN GOAL (Parameter), it was coming in Jenkins Console as a plain text.
Although I was using "Password Parameter" to pass the password at run-time but then also it was coming as plain text in console.
I tried to use the "secret text" to encrypt the password but then my job started failing because the encrypted password was getting passed to Maven Goals, which was not able to connect to DB.
Solution:
I used "Inject passwords to the build as environment variables" from Build Environment and defined its value as my "password parameter" (my password parameter name was db_password) which I am passing as parameter at run-time (eg.: I defined my inject password value as : ${db_password} ).
And this is working as expected. The password which I am passing while running my job is coming as [*******]
[console log:
Executing Maven: -B -f /work/jenkins_data/workspace/S2/database-deployment-via-flyway-EDOS/pom.xml clean compile -Ddb=UAT_cms_core -DdatabaseSchema=cms-core -Dmode=info -DdeploymentVersion=1.2.9 -Ddb_user=DB_USER -Ddb_password=[*******]
]
Related
I am searching for some help in how to configure Azure PostgreSQL DB in a Docker Swarm based Gitlab instance.
Initially, I followed the documentation in https://docs.gitlab.com/13.6/ee/administration/postgresql/external.html. Yet I came to find out that the default provided user is in the form of username, whereas Azure requires it to be in the form of username#hostname. I tried passing the username in the gitlab.rb file (gitlab_rails['db_username'] = 'username#hostname') but it still failed, even after replacing the # with the %40 as URI encoded.
After some extensive searching, I found this documentation - https://docs.gitlab.com/13.6/ee/administration/environment_variables.html, which suggests using the DATABASE_URL environment variable to set the full connection string in the form postgresql://username:password#hostname:port/dbname, which I did and it did solve the issue for Gitlab itself communicating with Azure PostgreSQL (in this case I replaced the username with username%40hostname, according to Azure requirements).
Allas, the success was short lived since then I came to find out that neither Puma and Sidekiq can connect to the database, always throwing the following error:
==> /var/log/gitlab/sidekiq/current <==
could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?
After some searching, I found that gitlab-ctl is generating the following file when starting the Gitlab instance:
# This file is managed by gitlab-ctl. Manual changes will be
# erased! To change the contents below, edit /etc/gitlab/gitlab.rb
# and run `sudo gitlab-ctl reconfigure`.
production:
adapter: postgresql
encoding: unicode
collation:
database: <database>
username: "<username>"
password:
host: "/var/opt/gitlab/postgresql"
port: 5432
socket:
sslmode:
sslcompression: 0
sslrootcert:
sslca:
load_balancing: {"hosts":[]}
prepared_statements: false
statement_limit: 1000
connect_timeout:
variables:
statement_timeout:
(database and username where removed)
Pretty much it ignores the DATABASE_URL env variable and assumes the now non-existing configuration parameters in gitlab.rb.
So, right now, I'm a bit out of options and was wondering if anyone has had a similar issue and, if so, how where you able to overcome this.
Any help is appreciated.
Thanks in advance.
TL/DR: Pass the username#hostname string directly into the gitlab_rails['db_username'] in double quotes. The documentation for connecting to an Azure PostgreSQL in the official Gitlab page is not correct.
So, after some searching and going deep into the Gitlab configuration, I came to find out that the issue is very specific and related with the usage of docker secrets.
In my gitlab.rb configuration file, in the database configuration part, I'm using the following:
### GitLab database settings
###! Docs: https://docs.gitlab.com/omnibus/settings/database.html
###! **Only needed if you use an external database.**
gitlab_rails['db_adapter'] = "postgresql"
gitlab_rails['db_encoding'] = "unicode"
gitlab_rails['db_database'] = File.read('/run/secrets/postgresql_database')
gitlab_rails['db_username'] = File.read('/run/secrets/postgresql_user')
gitlab_rails['db_password'] = File.read('/run/secrets/postgresql_password')
gitlab_rails['db_host'] = File.read('/run/secrets/postgresql_host')
gitlab_rails['db_port'] = File.read('/run/secrets/postgresql_port')
gitlab_rails['db_sslmode'] = 'require'
Now, this exact configuration was used previously for testing purposes and worked (but without the usage of Azure PostgreSQL database). And I'm passing the correct secrets to docker and I've confirmed that the secrets in fact, do exist.
(Sidenote: Also, I've established that Gitlab uses the method ActiveRecord::Base.establish_connection from the Ruby ActiveRecord::Base library in order to connect to the database)
Yet, when using the username#hostname configuration for the user and passing that into the postgresql_user secret, suddenly the ActiveRecord::Base.establish_connection method assumes that the #hostname is the actual hostname to where I want to connect to. And I've confirmed that the secret is being generated correctly inside the docker container
Now, it gets even stranger because if I pass the username#hostname string directly to the gitlab.rb file - gitlab_rails['db_username'] parameter - in double quotes, it suddenly starts connecting without complaining.
So, in short, if using an Azure PostgreSQL database for a dockerized Gitlab instance and using secrets to pass the configuration to the gitlab.rb file, don't pass the username#hostame through a secret, but put it directly in the gitlab.rb file.
I don't know if this is a specific issue of Ruby or of Gitlab (I'm not a Ruby developer), but I did try converting the File.read output to a String, to a symbol, used the File.open('filepath', &:readline) and other shenanigans, but nothing worked. So, if anyone out there would care to add their reason for this, please feel free to do so.
Also, the tutorial provided by Azure - https://learn.microsoft.com/pt-pt/azure/postgresql/connect-ruby - doesn't work with Gitlab, since it complains about the %40.
Hope this can help anyone out there.
My workflow goes like this :
I have a custom node module installed in my Jenkins slave that calls a few shell scripts . I would like to pass the credentials obtained through the credentials()
method provided by Jenkins declarative pipeline syntax.
What happens right now is that when I pass the username and password environment variables provided by the pipeline to the node module which in turn passes the arguments to the shell script which is used to CURL a file to a remote location
I get bad credentials failure.
But if I do the same thing by simply calling the shell script whithout passing thorugh the node_module the same creds work fine.
Is there any workaround for this behaviour ? I am just trying to understand here
Thanks in advance
So i am trying to figure out how to NOT display my password once its gets passed on to OS from cloudformation. So first i am using below on my cloudformation script, with "NoEcho" the password that i put in is started out...
"DBPASS" : {
"NoEcho" : "true",
"Description" : "Password for oracle user",
"Type" : "String",
"MinLength" : "1",
"MaxLength" : "20"
},
then in the user data section of my cloud formation i do below to set the password for oracle user. But the problem is that the user is echo'ed out to the boot.log/cloud-init.log so the password is visible...i am trying to hide the password so its not seen in the logs.
"DBPASS=",
{
"Ref": "DBPASS"
},
"\n",
"echo -e \"$DBPASS\n$DBPASS\" | passwd $oracle\n",
Then i was thinking of doing something like below but not sure how to pass in "DBPASS" variable to the input twice..
stty -echo
read DBPASS
stty echo
My Goal is to set the password for oracle user without echoing out to the logs...
If you need to protect sensitive information from being readable from within your EC2 instance, then you shouldn't put it in your user-data boot script at all, regardless of whether it's being stored as part of Cloud-init's default log output, because the user-data script will still always be readable as part of the instance metadata.
Refer to this Important note in the Instance Metadata and User Data section of the EC2 documentation:
Important
Although you can only access instance metadata and user data from within the instance itself, the data is not protected by cryptographic methods. Anyone who can access the instance can view its metadata. Therefore, you should take suitable precautions to protect sensitive data (such as long-lived encryption keys). You should not store sensitive data, such as passwords, as user data.
As one alternate approach to sensitive data, you could upload the content to a private S3 bucket, then download it to the EC2 instance using aws s3 cp from your user-data script. See my answer to the question, How can I (securely) download a private S3 asset onto a new EC2 instance with cloudinit? for more details on this approach.
You can pass your DB credentials as Parameter from the command line. You will need to pass those credentials while launching the Cloudformation stack but will not be visible anywhere. Check out this templatewhere DB parameter are provided from parameter ('default' is not set in parameters. So, you have to pass them while launching your cloudformation stack)
I'm trying to update Jenkins' root URL via the Groovy API, so I can script the deployment of a Jenkins master without manual input (aside: why is a tool as popular with the build/devops/automation community as Jenkins so resistant to automation?)
Based on this documentation, I believe I should be able to update the URL using the following script in the Script Console.
import jenkins.model.JenkinsLocationConfiguration
jlc = new jenkins.model.JenkinsLocationConfiguration()
jlc.setUrl("http://jenkins.my-org.com:8080/")
println(jlc.getUrl())
Briefly, this instantiates a JenkinsLocationConfiguration object; calls the setter setUrl with the desired value, http://jenkins.my-org.com:8080/; and prints out the new URL to confirm that it has changed.
The println statement prints what I expect it to, but following this, the value visible through the web interface at "Manage Jenkins" -> "Configure System" -> "Jenkins URL" has not updated as I expected.
I'm concerned that the value hasn't been update properly by Jenkins, which might lead to problems when communicating with external APIs.
Is this a valid way to fix the Jenkins root URL? If not, what is? Otherwise, why isn't the change being reflected in the config page?
You are creating a new JenkinsLocationConfiguration object, and updating the new one, not the existing one being used
use
jlc = JenkinsLocationConfiguration.get()
// ...
jlc.save()
to get the one from the global jenkins configuration, update it and save the config descriptor back.
see : https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/jenkins/model/JenkinsLocationConfiguration.java
I'm trying to config a SonarQube server using puppet.
My puppet manifests install software, deploy my custom sonar.properties, deploy ssl certificates, download and configure few plugins and, at last, start service.
The goal is config and reconfig SonarQube in automatic way.
During my postconfig step, I launch a puppet exec whith this SQL to set my own password form admin user.
"UPDATE users SET crypted_password='***********************************', salt='*******************************' where login='admin'
How I can calculate crypted_password and salt values for my password? (nowadays i use a fake sonar to change admin pass and look the value in db)
In pseudo code some like this...
crypted_password=crypt('pass')
Where crypt is
funcion crypt (anypass)
{
........
}
Thanks.
In the sonar-server's ruby source there is a ruby file for authentication by password: by_password.rb. Here you can see how Sonar encrypts passwords:
def password_digest(password, salt)
digest = REST_AUTH_SITE_KEY
REST_AUTH_DIGEST_STRETCHES.times do
digest = secure_digest(digest, salt, password, REST_AUTH_SITE_KEY)
end
digest
end
secure_digest is defined as:
def secure_digest(*args)
Digest::SHA1.hexdigest(args.flatten.join('--'))
end
So the encrypted password is the SHA1 of digest--salt--password--REST_AUTH_SITE_KEY repeated REST_AUTH_DIGEST_STRETCHES times. The values of REST_AUTH_SITE_KEY and REST_AUTH_DIGEST_STRETCHES are set in /web/WEB-INF/config/initializers/site_keys.rb and are empty string and 1 by default.
This is one way of achieving your goal. In my opinion a much better way is by creating a user via Sonar's REST API. However unfortunately it doesn't seem possible at the time (v4.1.2) to add a user to a group via the REST API.