I have a private key for deployment which I have set as a File Environment variable in the gitlab ci settings. However when I try and do
echo $VAR
the output is the entire contents of the key something like
----BEGIN PRIVATE KEY---
dfbvjladfbgvjksadbavkjabdfjnvkdjbnv;sdf98qw374yriuebglvkjsdfgydjkfvnldafkjbvgjkaldb
----END PRIVATE KEY---
I would like to have this key stored in a file and would like the path to the file as the environment variable $VAR. I am unsure of how to do this. Any help will be appreciated.
There was a couple of mistakes with respect to using the Environment variables.
The environment variables when declared in settings need not be imported in the yaml files as variables.
Mistake
variables:
VAR : $VAR
This leads to the contents of the file being exposed in the log which we dont want in this case.
Correct Way:
We can just use the key as is in the yaml with $VAR this should give a temporary path to the file where the key is stored.
Related
I have a bitbucket pipeline to push a docker image. I've defined the variable $DOCKERHUB_USERNAME=example
In my build step I have the line:
VERSION=$(npm run version --workspace=#example/core-web --silent)
When this runs though, its replacing #example with #$DOCKERHUB_USERNAME
VERSION=$(npm run version --workspace=#$DOCKERHUB_USERNAME/core-web --silent)
How can I escape that text so bitbucket doesn't try to replace it with the variable thats set to the same text? It just coincidentally is the same name, but they are not related.
If an environment variable is marked as a secret variable, Bitbucket activates a security feature that masks any accidental print of its value in the logs, replacing it by its variable name.
See https://support.atlassian.com/bitbucket-cloud/docs/variables-and-secrets/#Secured-variable-masking
Note this has no effect on the actual instructions being run: the value is only masked in the pipeline logs that are shown to you.
You should avoid such weak secrets. Using dictionary words that can legitimately show up in the logs will cause this security feature to expose the value of your secret so that it could be inferred even if it was never deliberately printed.
If you do not want to setup a secure value because it is not truly a secure variable, simply configure the variable as a regular public variable.
I am trying to edit yaml file to construct ci/cd in a gitlab project, troubled with the variables problem. I set a variable whose value is file path, and the number of file paths is multiple, so that I can use the words rules: changes: '$var'. But how can I set the variables with multi values? If I use the wildcards, just like this: variables: COMMON_FILE: ./{ci/*,camke/*,*.yml}, Then use rules: changes: '$COMMON_FILE' in the job, it will cause such error:"Local file'./ci/*' does not exist". If I reduce the number of file path, like this: COMMON_FILE: ./ci/*, it will not work yet. If the variable is like this: variables: COMMON_FILE: ci/*, it's ok. It seems that a file path with the prefix './' will not work. If I don't use wildcards, I must set more variables. Thus, is there any way that a variable could have multi values in the yaml file of gitlab ci/cd?
For example the file that I have is test.env
test.env has the content
export SAMPLE="true"
I want the variable SAMPLE to be set as a pipeline variable before running the pipeline
I am trying the below mentioned solution but it is not really helping
before_script:
- git clone test.env
- source test.env
stages:
- publish
test:
stage: publish
trigger:
project: test_pipeline
branch: master
strategy: depend
only:
variables:
- $SAMPLE == 'True'
Is there any way to source the variables in the before hand and then set the pipeline variables so that execution can happen based on those pipeline variables
Currently with Gitlab CI there's no way to provide a file to use as environment variables, at least not in the way you stated. There are a couple of other options however.
First is take all the individual variables you would have in your test.env file and store them as separate Secret Variables. You can set these by going to your project's settings, -> CI/CD, -> Variables. Environment Variables defined here will automatically be available in every pipeline job for this project (although you can select the Protect Variable checkbox, which will only make the variable available for pipelines on Protected Branches).
The next option is to copy the entire test.env file contents, go back to your project’s Secret Variables (as described above), but this time change the Variable Type to "File", and paste the file contents as the value. When you use a "File" type variable, Gitlab will create a temporary file in each of your pipeline jobs (again, unless you check the Protect Variable option). Then the path to that file will be stored as the env variable with the key you selected. This would allow you to do things like cat $my_file_variable, which would evaluate as cat /path/to/temporary/file, then cat the contents.
A final option which is closest to your original request, is to add a job before all your other jobs that would require the test.env file that looks like this:
stage: env_setup # or whatever
script:
- : # this is the bash Null Command that does nothing and always succeeds
artifacts:
reports:
dotenv: test.env
For this job, the only purpose is to turn your test.env file into environment variables. We don't need to do anything else with it, so we use the Null Command for the script section (since a job without at least the script section will fail). The artifacts part is the important stuff here. Gitlab supports a special Report type called dotenv that takes a single argument: a path to a file. The file will get uploaded as an artifact like any other, but for subsequent jobs (or those that use the dependencies keyword with this job name) instead of pulling down the artifact as a file, each item in test.env will be turned into an environment variable, so you can use it like $SAMPLE, etc.
Personally I prefer the first two options over the third, and of the first 2, the 2nd is the easiest as you just have to copy and paste the file you have now into a variable. The reason the third option isn't ideal is that it still allows you to have sensitive variables (like passwords) in your git repository, which isn't ideal from a security standpoint. Either of the first two options eliminate that problem.
I am using the Config crate in Rust, and would like to use environment variables to set keys inside a section of the config. The end goal is to override application settings from a docker compose file, or docker command line, using the environment.
If my config was the following, could I use a specifically crafted environment variable to set database.echo ?
(code blurb below is taken from this example)
debug = true
[database]
echo = true
The example code to configure this using the environment variables illustrates only to set keys at the top level. Wondering how to extend this. The .set() takes a hierarchical key, so I'm hopeful that there's a way to encode the path in the env variable name.
Answering my own question.
I just noticed that the Environment code accepts a custom separator which will get replaced with . (dot).
So one can set the separator to something like _XX_ and that would get mapped to a ".". Setting DATABASE_XX_ECHO=true, for instance would then change the database.echo key.
I use Terraform to provide some Google infrastructure. I would like to store some configuration variables in an external (non-terraform) config file. The idea is to use those variables in the Terraform and bash also, so I wouldn't like to use typical .tfvars file. How to achieve this?
I have got three files and let's assume for simplicity, that they are being stored in the same directory.
General configuration files with the variables to ingest:
# config.txt
GOOGLE_PROJECT_ID='my-test-name'
GOOGLE_REGION='my-region'
Terraform file with the datasources:
# datasources.tf
data "local_file" "local_config_file" {
filename = "./config.txt"
}
Terraform file with the variables:
# variables.tf
variable "project_id" {}
variable "region" {
default = 'europe-west3'
}
If all of your variables you'd like to use in Terraform are string-type variables, you can define them as environment variables to use them in Terraform and your Bash scripts:
Terraform will read environment variables in the form of TF_VAR_name to find the value for a variable. For example, the TF_VAR_region variable can be set in the shell to set the region variable in Terraform.
# config.sh
export TF_VAR_region="my-region"
export TF_VAR_project_id="my-test-name"
Note that this approach won't work for list or map type variables:
Note: Environment variables can only populate string-type variables. List and map type variables must be populated via one of the other mechanisms.
See the docs here for more information.