Given the following very simple .gitlab-ci.yml pipeline:
---
variables:
KEYCLOAK_VERSION: 20.0.1 # this should be populated from reading a file from the repo...
stages:
- test
build:
stage: test
script:
- echo "$KEYCLOAK_VERSION"
As you might see, this simply outputs the value of KEYCLOAK_VERSION defined in the variables section.
Now, the Git repository contains a env.properties file with KEYCLOAK_VERSION=20.0.1 as content. How would I read the variable from that file and use it in the GitLab pipeline?
The documentation mentions import but this seems to be using YAML files.
To read variables from a file you can use the source or . command.
script:
- source env.properties
- echo $KEYCLOAK_VERSION
Attention:
One reason why you might not want to do it this way is because whatever is in env.properties will be run in your shell, such as rm -rf /, which could be very dangerous.
Maybe you can take a look here for some other solutions.
I have a simple pipeline with one job to test bash scripts. The pipeline as follow:
image: alpine/git
stages:
- test_branching
test_branch:
stage: test_branching
before_script:
- mkdir -p .common
- wget https://x.x.x.x/branching.sh > .common/test.sh && chmod +x .common/test.sh
- source .common/test.sh
script:
- test_pipe
- echo "app version is ${app_version}"
The bash script as follow:
#!/bin/sh
function test_pipe () {
app_version="1.0.0.0-SNAPSHOT"
}
The problem is that the pipeline for whatever reason does not recognize the function inside the script. The logs are:
...
$ test_pipe
/scripts-1050-417479/step_script: eval: line 180: test_pipe: not found
Does anybody know what happend with this?? I miss a lot Jenkins shared libraries, gitlab does not have it, also gitlab does not have the function to include scripts inside yml files.
I dont want to use multiproject pipeline, I need to do it at this way. This is only an example of a more complicated pipeline logic.
Thanks in advance
As the documentation states before_script is just concatenated together with script and run on a single shell. The script you are downloading does not define test_pipe.
... gitlab does not have the function to include scripts inside yml
files.
It does, just use the YAML multiline literal syntax with |, e.g.:
script:
- |
echo "this"
echo "is"
echo "an \
example"
Here's what I am trying to do.
in .gitlab-ci:
Check schedules pass:
stage: check
image: ${myimage}
script:
- MY_CI_VAR=aVeryLongVariable
- echo "$MY_SCHEDULE_VAR"
In schedules:
Which is not working.
The reason I want to do this is for picking different variable (out of many in the job) on each schedule.
Yes, it is possible to use variables within other variables. This feature was released in GitLab 14.3.
However, since you are using GitLab 13.x, this feature won't be available to you.
You may be able to get around this limitation by using a static value and altering your job script accordingly.
myjob:
before_script: |
if [[ "$SCHEDULE_VAR" == "abc" ]]; then
export FOO="$MY_CI_VAR"
fi
# ...
In versions of GitLab < 14.3 you can still make use of other variables within variables, but instead by using $$ to preserve variables from evaluation.
Example from the docs:
variables:
FLAGS: '-al'
LS_CMD: 'ls "$FLAGS" $$TMP_DIR'
script:
- 'eval "$LS_CMD"' # Executes 'ls -al $TMP_DIR'
I have a gitlab yaml file for running certain jobs. In the variables part, I have declared certain variables with values and when I try to use it in another variable formation, it is actually generating but not fetching in the later part of job execution.
Code tried is as below:
variables:
env: "prod"
user: "test"
region: "us-east"
var1: '$env-$user-$region'
As suggested in one forum to include var1 formation in before_script script part. I tried it, but it was also not returning the var1 value correctly.
Any help will be appreciated.
At the bottom of this section of the official documentation, they describe using variables within variables:
You can use variables to help define other variables. Use $$ to ignore a variable name inside another variable:
variables:
FLAGS: '-al'
LS_CMD: 'ls $FLAGS $$TMP_DIR'
script:
- 'eval $LS_CMD' # Executes 'ls -al $TMP_DIR'
I was able to follow this pattern, and additionally I was combining variables in the script: step with a command such as:
script:
- APP_NAME=$APP_NAME-$VERSION
Hi I am trying to find out how to set environment variable with Ansible.
something that a simple shell command like this:
EXPORT LC_ALL=C
tried as shell command and got an error
tried using the environment module and nothing happend.
what am I missing
There are multiple ways to do this and from your question it's nor clear what you need.
1. If you need environment variable to be defined PER TASK ONLY, you do this:
- hosts: dev
tasks:
- name: Echo my_env_var
shell: "echo $MY_ENV_VARIABLE"
environment:
MY_ENV_VARIABLE: whatever_value
- name: Echo my_env_var again
shell: "echo $MY_ENV_VARIABLE"
Note that MY_ENV_VARIABLE is available ONLY for the first task, environment does not set it permanently on your system.
TASK: [Echo my_env_var] *******************************************************
changed: [192.168.111.222] => {"changed": true, "cmd": "echo $MY_ENV_VARIABLE", ... "stdout": "whatever_value"}
TASK: [Echo my_env_var again] *************************************************
changed: [192.168.111.222] => {"changed": true, "cmd": "echo $MY_ENV_VARIABLE", ... "stdout": ""}
Hopefully soon using environment will also be possible on play level, not only task level as above.
There's currently a pull request open for this feature on Ansible's GitHub: https://github.com/ansible/ansible/pull/8651
UPDATE: It's now merged as of Jan 2, 2015.
2. If you want permanent environment variable + system wide / only for certain user
You should look into how you do it in your Linux distribution / shell, there are multiple places for that. For example in Ubuntu you define that in files like for example:
~/.profile
/etc/environment
/etc/profile.d directory
...
You will find Ubuntu docs about it here: https://help.ubuntu.com/community/EnvironmentVariables
After all for setting environment variable in ex. Ubuntu you can just use lineinfile module from Ansible and add desired line to certain file. Consult your OS docs to know where to add it to make it permanent.
I did not have enough reputation to comment and hence am adding a new answer.
Gasek answer is quite correct. Just one thing: if you are updating the .bash_profile file or the /etc/profile, those changes would be reflected only after you do a new login.
In case you want to set the env variable and then use it in subsequent tasks in the same playbook, consider adding those environment variables in the .bashrc file.
I guess the reason behind this is the login and the non-login shells.
Ansible, while executing different tasks, reads the parameters from a .bashrc file instead of the .bash_profile or the /etc/profile.
As an example, if I updated my path variable to include the custom binary in the .bash_profile file of the respective user and then did a source of the file.
The next subsequent tasks won't recognize my command. However if you update in the .bashrc file, the command would work.
- name: Adding the path in the bashrc files
lineinfile: dest=/root/.bashrc line='export PATH=$PATH:path-to-mysql/bin' insertafter='EOF' regexp='export PATH=\$PATH:path-to-mysql/bin' state=present
- - name: Source the bashrc file
shell: source /root/.bashrc
- name: Start the mysql client
shell: mysql -e "show databases";
This would work, but had I done it using profile files the mysql -e "show databases" would have given an error.
- name: Adding the path in the Profile files
lineinfile: dest=/root/.bash_profile line='export PATH=$PATH:{{install_path}}/{{mysql_folder_name}}/bin' insertafter='EOF' regexp='export PATH=\$PATH:{{install_path}}/{{mysql_folder_name}}/bin' state=present
- name: Source the bash_profile file
shell: source /root/.bash_profile
- name: Start the mysql client
shell: mysql -e "show databases";
This one won't work, if we have all these tasks in the same playbook.
Here's a quick local task to permanently set key/values on /etc/environment (which is system-wide, all users, thus become is needed):
- name: populate /etc/environment
lineinfile:
path: "/etc/environment"
state: present
regexp: "^{{ item.key }}="
line: "{{ item.key }}={{ item.value}}"
with_items: "{{ os_environment }}"
become: yes
and the vars for it:
os_environment:
- key: DJANGO_SETTINGS_MODULE
value : websec.prod_settings
- key: DJANGO_SUPER_USER
value : admin
and, yes, if you ssh out and back in, env shows the new environment variables.
p.s. It used to be dest as in:
dest: "/etc/environment"
but see the comment
For the task only: inlining works, some of the time.
——————-
Note: the stuff below is more an observation/experiment than a recommendation.
——————-
The first task is the equivalent to Michael's top voted answer.
The second doesn't work, but then again foo=1 echo $foo doesn't work in bash either (I suspect that's because echo is a builtin).
The third does work, as it does in bash, and takes very little effort. However... when I tried doing this to set a node variable, it failed miserably until I used Michael's answer.
tasks:
- name: Echo my_env_var
shell: "echo $MY_ENV_VARIABLE"
environment:
MY_ENV_VARIABLE: value1
- name: Echo my_env_var inline, doesnt work in bash either
shell: "MY_ENV_VARIABLE=value2 echo $MY_ENV_VARIABLE"
- name: set my_env_var inline then env
shell: "MY_ENV_VARIABLE=value3 env | egrep MY_ENV"
output:
TASK [Echo my_env_var] *********************************************************
changed: [192.168.63.253] => changed=true
cmd: echo $MY_ENV_VARIABLE
stdout: value1
TASK [Echo my_env_var inline, doesnt work in bash either] **********************
changed: [192.168.63.253] => changed=true
cmd: MY_ENV_VARIABLE=value2 echo $MY_ENV_VARIABLE
stdout: ''
TASK [set my_env_var inline then env] ******************************************
changed: [192.168.63.253] => changed=true
cmd: MY_ENV_VARIABLE=value3 env | egrep MY_ENV
stdout: MY_ENV_VARIABLE=value3
This is the best option. As said Michal Gasek (first answer), since the pull request was merged (https://github.com/ansible/ansible/pull/8651),
we are able to set permanent environment variables easily by play level.
- hosts: all
roles:
- php
- nginx
environment:
MY_ENV_VARIABLE: whatever_value
For persistently setting environment variables, you can use one of the existing roles over at Ansible Galaxy. I recommend weareinteractive.environment.
Using ansible-galaxy:
$ ansible-galaxy install weareinteractive.environment
Using requirements.yml:
- src: franklinkim.environment
Then in your playbook:
- hosts: all
sudo: yes
roles:
- role: franklinkim.environment
environment_config:
NODE_ENV: staging
DATABASE_NAME: staging
I'm installing krew plugin manager and some of its plugin using ansible for a Fish shell. To install the plugins, I wanted to use the $PATH value as set by my previous task in /.config/fish/config.fish.
Error
Using shell module with executable parameter was throwing:
stderr: |-
error: unknown command "krew" for "kubectl"
error: unknown command "krew" for "kubectl"
error: unknown command "krew" for "kubectl"
Solution
add a line declaring the path to the krew bin in /.config/fish/config.fish
- name: Add krew to $PATH
lineinfile:
path: '{{ home }}/.config/fish/config.fish'
search_string: krew
line: set --append --export --global PATH $HOME/.krew/bin
Then, use the shell module to source Fish config file and run kubectl commands to install my plugins:
- name: Install `kubectx` (context) and `kudens` (namespace)
shell: |
source {{ home }}/.config/fish/config.fish
kubectl krew install ctx
kubectl krew install ns
kubectl krew install oidc-login
args:
executable: /usr/bin/fish
Disclaimer
Look a bit hack-ish to me, would love more ansible-ish solution.
You can also set env with a custom become plugin
hosts setting
ansible_become=yes
ansible_become_method=foo
become_plugins/foo.py
from ansible.plugins.become import BecomeBase
class BecomeModule(BecomeBase):
def build_become_command(self, cmd, shell):
cmd = 'PYTHONPATH="/foo/bar:$PYTHONPATH" ' + cmd
return cmd