Excluding gitlab-ci.yml from merge-request - gitlab

I have 2 branches: stage and master - and each of them has its own .gitlab-ci.yml with different scripts. How is it possible to exclude these files from merge-request, so that they remain unique for each of the branches?
i tried with gitattributes and merge strategy - but this method only works locally. Then I decided to create one common gitlab-ci.yml for all branches, but in this case it is necessary to set the conditions for the execution of different jobs for different branches. I tried in the following way:
runTests_job:
stage: runTests
tags:
- ps
rules:
- if: $CI_COMMIT_BRANCH == "stage"
script:
- chcp 65001
- Invoke-Sqlcmd -ConnectionString $CONNSTR -Query "EXEC dbo.test"
but the job was still running in the master branch. Also tried:
runTests_job:
stage: runTests
tags:
- ps
only:
refs:
- stage
script:
- chcp 65001
- Invoke-Sqlcmd -ConnectionString $CONNSTR -Query "EXEC dbo.test"
but it didn't work. Maybe I made a mistake in the syntax somewhere?

You can use make custom variables and conditions when the pipeline should run.
You can check from which branch the commit is been made and the accordingly you can check and run the script.
variables:
DEPLOY_VARIABLE: "default-deploy"
workflow:
rules:
- if: $CI_COMMIT_REF_NAME == $CI_DEFAULT_BRANCH # master
variables:
DEPLOY_VARIABLE: "deploy-master" # Override DEPLOY_VARIABLE
- when: always # Run pipeline in other cases
job1:
rules:
- if: $DEPLOY_VARIABLE == "deploy-master"
# run the script
script:
- chcp 65001
- Invoke-Sqlcmd -ConnectionString $CONNSTR -Query "EXEC dbo.test"
job2:
rules:
- if: $DEPLOY_VARIABLE == "deploy-staging"
# run the script
script:
- chcp 65001
- Invoke-Sqlcmd -ConnectionString $CONNSTR -Query "EXEC dbo.test"
Just a heads-up the syntax could be a bit wrong. But you can use the concept to proceed further.
You might also want to refer to the docs: https://docs.gitlab.com/ee/ci/yaml/

Related

how to pass variables between gitlab-ci jobs?

I have a gitlab-ci like this:
stages:
- calculation
- execution
calculation-job:
stage: calculation
script: ./calculate_something_and_output_results.sh
tags:
- my-runner
execution-job:
stage: execution
script: ./execute_something_with_calculation_results.sh foo
tags:
- my-runner
The foo argument in execution-job is base on the results of calculation-job. I want to pass the results from one job to another job via variables. How can I do that?
If you're looking to get the results without storing a file anywhere you can use artifacts: reports: dotenv. This is taken entirely from DarwinJS shared-variables-across-jobs repo.
stages:
- calculation
- execution
calculation-job:
stage: calculation
script: - |
# stores new or updates existing env variables, ex. $OUTPUT_VAR1
./calculate_something_and_output_results.sh >> deploy.env
tags:
- my-runner
artifacts:
reports:
#propagates variables into the pipeline level, but never stores the actual file
dotenv: deploy.env
execution-job:
stage: execution
script: - |
echo "OUTPUT_VAR1: $OUTPUT_VAR1"
./execute_something_with_calculation_results.sh foo
tags:
- my-runner
AFAIK it is not possible to pass a variable directly from one job to another job. Instead you have to write them into a file and pass that as artifact to the receiving job. To make parsing of the file easy, I recommend to create it with bash export statements and source it in the receiving job's script:
calculation-job:
stage: calculation
script:
- ./calculate_something_and_output_results.sh
- echo "export RESULT1=$calculation_result1" > results
- echo "export RESULT2=$calculation_result2" >> results
tags:
- my-runner
artifacts:
name: "Calculation results"
path: results
execution-job:
stage: execution
script:
- source ./results
# You can access $RESULT1 and $RESULT2 now
- ./execute_something_with_calculation_results.sh $RESULT1 $RESULT2 foo
tags:
- my-runner
needs: calculation-job
Note the ./ when sourcing results might be necessary in case of a POSIX compliant shell that does not source files in the current directory directly like, for example, a bash started as sh.
As a simpler version of what #bjhend answered (no need for export or source statements), since GitLab 13.1 the docs. recommend using a dotenv artifact.
stages:
- calculation
- execution
calculation-job:
stage: calculation
script:
# Output format must feature one "VARIABLE=value" statement per line (see docs.)
- ./calculate_something_and_output_results.sh >> calculation.env
tags:
- my-runner
artifacts:
reports:
dotenv: calculation.env
execution-job:
stage: execution
script:
# Any variables created by above are now in the environment
- ./execute_something_with_calculation_results.sh
tags:
- my-runner
# The following is technically not needed, but serves as good documentation
needs:
job: calculation-job
artifacts: true
If you have a job after the calculation stage that you don't want to use the variables, you can add the following to it:
needs:
job: calculation-job
artifacts: false

gitlab not expanding variable used in parallel/matrix

I am trying to pass the variable to parallel/matrix and do not see that getting expanded and the job failing. This is being set in the job from the environment variable. I am trying to echo the variable in script and see it shows the right value, but does not get substituted in parallel/matrix. Am I missing anything?
.common_deploy:
script:
- |
echo "showing the regions from environment"
echo $qa_regions
echo "showing the regions from job variable"
echo $REGIONS
parallel:
matrix:
- REGION: "${REGIONS}"
DeployToQA:
variables:
ENVIRONMENT: qa
REGIONS: $qa_regions
extends:
- .common_deploy
stage: deploy
rules:
- if: '$CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == "master"'
allow_failure: true
Here the variable $qa_regions has the value of "us-west-2,us-east-1", I was expecting to see the 2 jobs for those regions , but i am seeing the job as DeployToQA: [${REGIONS}]
Variable expansion for the parallel keyword is currently not supported. There is an open issue for this request.
You can take a look at the documentation where variables can be used.

How to run Gitlab CI only for specific branches and some rules?

I am trying to build ci according to the following criteria:
Run only when pushing to master branch - already implemented
Run on commit [run ci] - already implemented
Manual launch from the Run pipeline - already implemented
do not launch when pushing to the master branch when only Tag changes
do not start in other cases if they do not fall under the first 3 options
The tag may change without changes in other files and do not need to run the assembly, I tried this
if: $CI_COMMIT_BRANCH == "master" && $CI_COMMIT_TAG =~ /^$/
but it doesn't work if there are changes in both files and tags.
I will be glad to help
My .gitlab-ci.yml
image: busybox:latest
build1:
stage: build
rules:
- if: $CI_COMMIT_MESSAGE =~ /\[run ci\]/
- if: $CI_COMMIT_BRANCH == "master"
- when: manual
allow_failure: false
script:
- echo "Do your build here"
test1:
stage: test
rules:
- if: $CI_COMMIT_MESSAGE =~ /\[run ci\]/
- if: $CI_COMMIT_BRANCH == "master"
- allow_failure: false
script:
- echo "Do a test here"
- echo "For example run a test suite"
deploy:
stage: deploy
rules:
- if: $CI_COMMIT_MESSAGE =~ /\[run ci\]/
- if: $CI_COMMIT_BRANCH == "master"
- allow_failure: false
script:
- echo "Do a test here"
- echo "For example run a test suite"

How should use variables?

If I add variables(build stage), the stages does not work. Where exactly am I going wrong. If remove the variables under only variables section, it runs.
build:
stage: build
script:
- echo "Build is running"
only:
changes:
- Dockerfile
- requirements.txt
- ./configs/*
variables:
- $BUILD == "True"
development:
stage: development
script:
- echo "development"
except:
variables:
- $BUILD == "False"
development_build:
stage: development_build
script:
- echo "BUILD OK"
rules:
- if: $BUILD == "True"
when: always
The documentation mentions
only:variables / except:variables
Use the only:variables or except:variables keywords to control when to add jobs to a pipeline, based on the status of CI/CD variables.
Keyword type: Job keyword. You can use it only as part of a job.
Possible inputs: An array of CI/CD variable expressions.
First: try and use rules:if to see if the issue persists.
job1:
variables:
VAR1: "variable1"
script:
- echo "Test variable comparison"
rules:
- if: $VAR1 == "variable1"
Second, try with a different variable name, in case BUILD is somehow reserved.
Check also where you have BUILD defined (is there a workflow:rules:variables section?)

how to build a rule which is based on dotenv variable which was defined previously in same pipeline

I'm setting a variable in a job in the pipeline and use it in the artifacts / dotenv.
It doesn't seem to work to use the variable as a condition in the rules.
How can I set a env variable and use it in another Job's rule?
Thanks a lot for your thoughts!
include:
- local: '/gitlab/cicd/.gitlab-ci_test.yml'
stages:
- build
- test
build_rule:
stage: build
rules:
- if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_MESSAGE =~ /R::/m
script:
- echo "Hello World"
- |
echo "RUNTESTS=1" > gitlabcicd.env
artifacts:
reports:
dotenv: gitlabcicd.env
test_always:
stage: test
script:
- echo "TestEnv" $RUNTESTS
# prints TestEnv 1
test_sometimes:
stage: test
rules:
- if: $RUNTESTS
# $RUNTESTS == "1" doesnt work either
script:
- echo "Runs only if variable was set"
# doesnt run, even if it prints above
As I posted in a comment before, but just so people can find it more easily, the answer is that, unfortunately, right now it is not possible to do that.
There is an issue about it here: https://gitlab.com/gitlab-org/gitlab/-/issues/235812
No satisfactory workaround that I'm aware of.
Is needed to load the .env file on every job
"Set variables":
stage: preparation
script:
- echo version=1.1 > varfile
- echo foo=bar >> varfile
artifacts:
paths:
- varfile
Dothings:
stage: run
script:
- load varfile
- echo ${foo}

Resources