image: python:3.7.3
pipelines:
branches:
Server:
- step:
name: Test
script:
- echo $MYTEST
My pipeline shows this:
echo $MYTEST
+ echo $MYTEST
$MYTEST
The variable is a "Repository variable" and is named MYTEST
I have done the same process with a node image and it is working fine. Because this is a python project i took python, but the environment variables don't seem to work.
I suppose, i cannot use a pure python docker image, if i need some bash commands, correct?
Related
Im trying pass a user defined variable from a custom pipeline to a step defined within the definitions section.
My yml snippet is below:
definitions:
steps:
- step: &TagVersion
trigger: manual
script:
- export VERSION=$VERSION
- echo "VERSION $VERSION"
custom:
run_custom:
- variables:
- name: VERSION
- step:
script:
- echo "starting"
- parallel:
- step:
<<: *TagVersion
variables:
VERSION: $VERSION
When I build the pipeline I can see the variable is listed as a pipeline variable when running the step TagVersion, and the correct value is shown there but dont know how to use this within the scripts section where im trying to echo out the value.
thanks
I'd like to parameterize the environment URL of one of my GitLab CI job in a project which belongs to a subgroup.
If for example I have:
CI_PROJECT_PATH = mygroup/mysubgroup/myproject
CI_PROJECT_NAMESPACE = mygroup/mysubgroup
CI_PROJECT_NAME = myproject
I'd like to have the URL to be something like https://mygroup.gitlab.com/-/mysubgroup/myproject/-/jobs/12345/artifacts/public/index.html.
But I cannot find a way to do this since there is no predefined variable for the "sub-namespace" (here mysubgroup) and there is no variable substitution in environment.url as far as I can see.
I tried in my gitlab-ci.yml things like that:
build:
stage: build
image: bash:latest
script:
- export # print the available variables
environment:
name: test
url: ${CI_SERVER_PROTOCOL}://${CI_PROJECT_ROOT_NAMESPACE}.${CI_PAGES_DOMAIN}/-/${CI_PROJECT_PATH#${CI_PROJECT_ROOT_NAMESPACE}/}/-/jobs/$CI_JOB_ID/artifacts/public/index.html
but the result is https://mygroup.gitlab.com/-/${CI_PROJECT_PATH#myproject/}/-/jobs/12345/artifacts/public/index.html.
References:
https://stackoverflow.com/a/58402821/1064669
https://gitlab.com/gitlab-org/gitlab/-/issues/350902
job:environment:url is directly processed by gitlab, which does not support this kind of parameter expansion.
You'll need to use bash or a similar shell for this to work as intended:
build:
stage: build
image: bash:latest
script:
- echo "DEPLOY_URL=${CI_SERVER_PROTOCOL}://${CI_PROJECT_ROOT_NAMESPACE}.${CI_PAGES_DOMAIN}/-/${CI_PROJECT_PATH#${CI_PROJECT_ROOT_NAMESPACE}/}/-/jobs/$CI_JOB_ID/artifacts/public/index.html" > deploy.env
- cat deploy.env
artifacts:
reports:
dotenv: deploy.env
environment:
name: test
url: $DEPLOY_URL
Based on the example given here: https://docs.gitlab.com/ee/ci/environments/#example-of-setting-dynamic-environment-urls
I think you need to escape the inner curly braces like this: (${CI_PROJECT_ROOT_NAMESPACE} should become $`{CI_PROJECT_ROOT_NAMESPACE`})
So the whole thing becomes:
url: ${CI_SERVER_PROTOCOL}://${CI_PROJECT_ROOT_NAMESPACE}.${CI_PAGES_DOMAIN}/-/${CI_PROJECT_PATH#$`{CI_PROJECT_ROOT_NAMESPACE`}/}/-/jobs/$CI_JOB_ID/artifacts/public/index.html
I've got some trouble defining a job. I'm in an early stage of defining my pipeline and I'm trying to solve my problems with variables :). I've got a few subfolder under in gcp/live/development and I'm trying to cd them and then run terraform stuff:
.plan-development:
variables:
ENVIRONMENT: "development"
TERRAFORM_DEVELOPMENT_PATH: "gcp/live/development"
TF_ROOT: ${TERRAFORM_DEVELOPMENT_PATH}/${CI_JOB_NAME%-plan-development}
stage: plan-development
script:
- cd ${TF_ROOT}
- gitlab-terraform plan
- gitlab-terraform plan-json
artifacts:
name: plan-${ENVIRONMENT}-${CI_JOB_NAME%-plan-development}
paths:
- ${TF_ROOT}/plan.cache
reports:
terraform: ${TF_ROOT}/plan.json
But according to my failing pipeline, the first script part cd ${TF_ROOT} doesn't enter the expected folder at gcp/live/development/cloud-nat.
I've got some debug outout, which tells me following:
$ cd ${TF_ROOT}
$ pwd && ls
/builds/b79ya_1j/0/devops/terraform/gcp/live/development
cloud-nat
firewall
gke
vpc
$ echo "${CI_JOB_NAME%-plan-development}"
cloud-nat
Am I missing something combing two variables (TF_ROOT: ${TERRAFORM_DEVELOPMENT_PATH}/${CI_JOB_NAME%-plan-development})?
Hope you can help me out.
Am I missing something combing two variables (TF_ROOT:
${TERRAFORM_DEVELOPMENT_PATH}/${CI_JOB_NAME%-plan-development})?
The issue is that Gitlab in variables section, doesn't perform bash operations, like you attempt here
${CI_JOB_NAME%-plan-development}
Gitlab literally searches for the variable named CI_JOB_NAME%-plan-development
That's why when it creates the TF_ROOT variable its value is gcp/live/development/ since it could not find the variable named CI_JOB_NAME%-plan-development
I am developing a Laravel project and I want to have CI with it. I am using bitbucket pipelines to do this. and I am using a ubuntu VPS.I would like to have separate folders for release-branchname.
for example, if I create a branch release-1.0.0 in bitbucket, and when i am trying to deploy it should create a folder in my remote server projectname/releases/release-1.0.0. I tried many ways but I was not successful.
here is my pipeline script
release-*:
- step:
name: Preparing pipeline for release
script:
- echo 'Preparing pipeline for releases'
- step:
name: Deploing release branches
trigger: manual
deployment: test
script:
- cat ./deploy.sh | ssh root#X.X.X.X
and here is my deploy.sh
echo 'Deployment started'
cd /home/core-cms/project-root
mkdir /home/core-cms/$BITBUCKET_BRANCH
git stash
git pull origin master
composer install
echo 'Deploy finished'
exit;
if I could pass $BITBUCKET_BRANCH variable into this deploy.sh, it would work perfectly I guess.
maybe I am totally wrong and there is another way to accomplish this. if so can anyone guide me, please? Thank you.
There are two different things that you are attempting to do:
Get name of the branch from the commit and pass it to the next step
default:
- step:
deployment: test
script:
- ssh user#host "bash -s" -- < ./deploy.sh $BITBUCKET_BRANCH
Get the branch name in your scrip and do the rest
BITBUCKET_BRANCH=${1}
echo 'Deployment started'
cd /home/core-cms/
git clone -b $BITBUCKET_BRANCH user#repository.com
composer install
echo 'Deploy finished'
exit;
Here is what I have done in the code in steps:
The $BITBUCKET_BRANCH is an standard Bitbucket variable. Be aware that this var is not available using custom or tag trigers.
In the pipeline code call the script on local machine pass the branch name to it and execute it on ssh server. The code ssh user#host "bash -s" -- < will run the rest of the code as if it is running it as a local code. Adding -- will stop the remote bash to take rest of the arguments for itself.
By adding the $BITBUCKET_BRANCH after the script file name, we pass it to the script.
In the script we get the input argument using $1 and assign it to $BITBUCKET_BRANCH variable.
The code will clone the code in the sub-directory. Here you might
need some changes to avoid git conflicts.
P.S. Just to mention, I haven't test the code. It might have some misspellings. However, the solution is there. You might be able to alter it to what you'd like to do.
Help, please. I have problems when using the CI tool.
Here's my .gitlab-ci.yaml
stages:
- test
test:
stage: test
environment:
name: test
url: https://word.mymusise.com/env_test.txt
script: echo "Running tests TEST=$TEST"
And I've define the test environment in EnvDocker > Pipelines > Environments
But it didn't export the environment from https://word.mymusise.com/env_test.txt in the CI job.
Running with gitlab-runner 11.4.2 (cf91d5e1)
on gitlab-ci runner a0e18516
Using Docker executor with image ubuntu:16.04 ...
Pulling docker image ubuntu:16.04 ...
Using docker image sha256:2a697363a8709093834e852b26bedb1d85b316c613120720fea9524f0e98e4a2 for ubuntu:16.04 ...
Running on runner-a0e18516-project-123-concurrent-0 via gitlab...
Fetching changes...
HEAD is now at d12c05b Update .gitlab-ci.yml
From https://gitlab.kingdomai.com/mymusise/envdocker
d12c05b..1a3954f master -> origin/master
Checking out 1a3954f8 as master...
Skipping Git submodules setup
$ echo "Running tests TEST=$TEST"
Running tests TEST=
Job succeeded
I define export TEST="test" in https://word.mymusise.com/env_test.txt, but it seems not working.
What should I do... Orz
Gitlab version: 11.4.0-ee
You want to run commands that are inside the text file that is accessible via http protocol.
With curl you can download the file and print it on curl's standard output. With command substitution $() you can grab the standard output. Then you can execute the commands itself (very unsafe, there might be multiple escaping issues).
script:
- $(curl "$url")
- echo "Running tests TEST=$TEST"
A safer alternative would be to just download the file and execute/source it.
script:
- curl "$url" > ./run_this.sh
# don't forget to add executable right to the file ;)
- chmod +x ./run_this.sh
- source ./run_this.sh
# pick out the trash
- rm ./run_this.sh
# rest of your script.
- echo "Running tests TEST=$TEST"
Downloading a shell script and executing it is a popular way of automating tasks, usually with curl url | bash. It is not supported "natively" by gitlab and I don't think it should be.