I want to use arrays in variables of my gitlab ci/cd yml file, something like that:
variables:
myarrray: ['abc', 'dcef' ]
....
script: |
echo myarray[0] myarray[1]
But Lint tells me that file is incorrect:
variables config should be a hash of key value pairs, value can be a hash
I've tried the next:
variables:
arr[0]: 'abc'
arr[1]: 'cde'
....
script: |
echo $arr[0] $arr[1]
But build failed and prints out bash error:
bash: line 128: export: `arr[0]': not a valid identifier
Is there any way to use array variable in .gitlab-ci.yml file?
According to the docs, this is what you should be doing:
It is not possible to create a CI/CD variable that is an array of values, but you can use shell scripting techniques for similar behavior.
For example, you can store multiple variables separated by a space in a variable, then loop through the values with a script:
job1:
variables:
FOLDERS: src test docs
script:
- |
for FOLDER in $FOLDERS
do
echo "The path is root/${FOLDER}"
done
After some investigations I found some surrogate solution. Perhaps It may be useful for somebody:
variables:
# Name of using set
targetType: 'test'
# Variables set test
X_test: 'TestValue'
# Variables set dev
X_dev: 'DevValue'
# Name of variable from effective set
X_curName: 'X_$targetType'
.....
script: |
echo Variable X_ is ${!X_curName} # prints TestValue
Another approach you could follow is two use a matrix of jobs that will create a job per array entry.
deploystacks:
stage: deploy
parallel:
matrix:
- PROVIDER: aws
STACK: [monitoring, app1]
- PROVIDER: gcp
STACK: [data]
tags:
- ${PROVIDER}-${STACK}
Here is the Gitlab docs regarding matrix
https://docs.gitlab.com/ee/ci/jobs/job_control.html#run-a-one-dimensional-matrix-of-parallel-jobs
Related
Given the following very simple .gitlab-ci.yml pipeline:
---
variables:
KEYCLOAK_VERSION: 20.0.1 # this should be populated from reading a file from the repo...
stages:
- test
build:
stage: test
script:
- echo "$KEYCLOAK_VERSION"
As you might see, this simply outputs the value of KEYCLOAK_VERSION defined in the variables section.
Now, the Git repository contains a env.properties file with KEYCLOAK_VERSION=20.0.1 as content. How would I read the variable from that file and use it in the GitLab pipeline?
The documentation mentions import but this seems to be using YAML files.
To read variables from a file you can use the source or . command.
script:
- source env.properties
- echo $KEYCLOAK_VERSION
Attention:
One reason why you might not want to do it this way is because whatever is in env.properties will be run in your shell, such as rm -rf /, which could be very dangerous.
Maybe you can take a look here for some other solutions.
Here's what I am trying to do.
in .gitlab-ci:
Check schedules pass:
stage: check
image: ${myimage}
script:
- MY_CI_VAR=aVeryLongVariable
- echo "$MY_SCHEDULE_VAR"
In schedules:
Which is not working.
The reason I want to do this is for picking different variable (out of many in the job) on each schedule.
Yes, it is possible to use variables within other variables. This feature was released in GitLab 14.3.
However, since you are using GitLab 13.x, this feature won't be available to you.
You may be able to get around this limitation by using a static value and altering your job script accordingly.
myjob:
before_script: |
if [[ "$SCHEDULE_VAR" == "abc" ]]; then
export FOO="$MY_CI_VAR"
fi
# ...
In versions of GitLab < 14.3 you can still make use of other variables within variables, but instead by using $$ to preserve variables from evaluation.
Example from the docs:
variables:
FLAGS: '-al'
LS_CMD: 'ls "$FLAGS" $$TMP_DIR'
script:
- 'eval "$LS_CMD"' # Executes 'ls -al $TMP_DIR'
I am trying to parallelize flutter build using GitLab using GitLab's parallel keyword and flutter's --total-shards and --shard-index.
Something like below
test_job:
stage: test
parallel: 3
script:
- flutter test --total-shards $CI_NODE_TOTAL --shard-index $CI_NODE_INDEX
However, this script fails in the last job because off-by-one error $CI_NODE_INDEX > $CI_NODE_TOTAL. Seems like it is undocumented that $CI_NODE_INDEX starts from 1 instead of 0.
I wanted to subtract the variables by using VARIABLES to $CI_NODE_INDEX_ZERO because the variable is being used multiple times throughout this long job (the script in the example above is shortened).
I tried this.
test_job:
stage: test
parallel: 3
variables:
CI_NODE_INDEX_ZERO: $( expr $CI_NODE_INDEX - 1 )
script:
- flutter test --total-shards $CI_NODE_TOTAL --shard-index $CI_NODE_INDEX_ZERO
The script still fails since the value of $CI_NODE_INDEX_ZERO is literal string expr $CI_NODE_INDEX - 1 instead of 0 (or whatever integer value needed).
This actually works in my local terminal.
petrabarus#Petras-Air % CI_NODE_INDEX_ZERO=5
petrabarus#Petras-Air % CI_NODE_INDEX=5
petrabarus#Petras-Air % echo $CI_NODE_INDEX
5
petrabarus#Petras-Air % CI_NODE_INDEX_ZERO=$( expr $CI_NODE_INDEX - 1 )
petrabarus#Petras-Air % echo $CI_NODE_INDEX_ZERO
4
How do I fix this?
Variables can only be literal values -- they are not evaluated in any way, like what happens in your bash shell.
If you want to use bash to evaluate and set variables for jobs, you can do that using a dotenv artifact.
make_variables:
stage: .pre # run before all jobs
script:
# evaluate the value of a variable
- DYNAMIC_VARIABLE=$(my-script)
# Add the value to a dotenv file.
- echo "DYNAMIC_VARIABLE=$DYNAMIC_VARIABLE" >> myvariables.env
artifacts:
reports: # set the variables for subsequent jobs
dotenv: myvariables.env
my_job:
script:
- echo "$DYNAMIC_VARIABLE"
Though the easier thing to do would be just to evaluate it directly in your script:
script:
- SHARD_INDEX=$( expr $CI_NODE_INDEX - 1 )
- flutter test --total-shards $CI_NODE_TOTAL --shard-index $SHARD_INDEX
i think because VARIABLES can't read from Gitlab CI
please check it https://docs.gitlab.com/ee/ci/variables/
example
test_variable:
stage: test
script:
- echo "$CI_JOB_STAGE"
please tried it
script:
- flutter test --total-shards "$CI_NODE_TOTAL" --shard-index "$CI_NODE_INDEX"
I have a gitlab yaml file for running certain jobs. In the variables part, I have declared certain variables with values and when I try to use it in another variable formation, it is actually generating but not fetching in the later part of job execution.
Code tried is as below:
variables:
env: "prod"
user: "test"
region: "us-east"
var1: '$env-$user-$region'
As suggested in one forum to include var1 formation in before_script script part. I tried it, but it was also not returning the var1 value correctly.
Any help will be appreciated.
At the bottom of this section of the official documentation, they describe using variables within variables:
You can use variables to help define other variables. Use $$ to ignore a variable name inside another variable:
variables:
FLAGS: '-al'
LS_CMD: 'ls $FLAGS $$TMP_DIR'
script:
- 'eval $LS_CMD' # Executes 'ls -al $TMP_DIR'
I was able to follow this pattern, and additionally I was combining variables in the script: step with a command such as:
script:
- APP_NAME=$APP_NAME-$VERSION
Is there any way to use an export variable, defined in the generic before_script:
before_script:
- export UPPERHASH=$(echo $CI_COMMIT_REF_SLUG | md5sum | tr [a-z] [A-Z])
into another job as a variable, because I am gonna use trigger but trigger does not allow to have any script, ex:
test variables:
stage: test-variables
variables:
UPPERHASH_TEST1: $UPPERHASH
trigger:
project: "...\..."
I have tried multiple options but none of them is working.
It will not work this way because "test variables".variables is processed before before_script
You only can refer to this variable in a script:
test variables:
stage: test-variables
script:
UPPERHASH_TEST1=$UPPERHASH
... trigger other project from command line ...
Read here on how to trigger other project from command line
https://docs.gitlab.com/ee/ci/triggers/README.html