I am trying to setup notifications to my Rocket.Chat server through my .gitlab-ci.yml file. I have my test and deploy stages working, but my notify stage is erroring out. I followed the instructions from here, but I adjusted the notify scripts to work with Rocket.Chat instead of Pushbullet.
Here is my .gitlab-ci.yml:
stages:
- test
- deploy
- notify
test:
stage: test
image: homeassistant/amd64-homeassistant
script:
- hass --script check_config -c .
deploy:
stage: deploy
only:
- master
before_script:
- 'which ssh-agent || ( apk update && apk add openssh-client )'
- eval $(ssh-agent -s)
- echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- echo "$SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts
- chmod 644 ~/.ssh/known_hosts
script:
- ssh $DEPLOY_USER#$DEPLOY_HOST "cd '$DEPLOY_PATH'; git pull; sudo systemctl restart home-assistant#homeassistant"
notify_success:
stage: notify
allow_failure: true
only:
- master
script:
- curl -X POST -H 'Content-Type: application/json' --data '{"text":"New Hass config deployed successfully!"}' https://chat.bryantgeeks.com/hooks/$ROCKET_CHAT_TOKEN
notify_fail:
stage: notify
allow_failure: true
only:
- master
when: on_failure
script:
- curl -X POST -H 'Content-Type: application/json' --data '{"text":"New Hass config failed. Please check for errors!"}' https://chat.bryantgeeks.com/hooks/$ROCKET_CHAT_TOKEN
I get this error in the CI Lint:
Status: syntax is incorrect
Error: jobs:notify_success:script config should be a string or an array of strings
If I change the notify script lines to have single quotes around it ('), I get the following error in CI Lint:
Status: syntax is incorrect
Error: (): did not find expected key while parsing a block mapping at line 33 column 7
If I try double quotes around the script line ("), I get the following error:
Status: syntax is incorrect
Error: (): did not find expected '-' indicator while parsing a block collection at line 33 column 5
I'm not sure what else to try or where to look at this point for how to correct this. Any help is appreciated.
YAML really doesn't like :'s in strings. The culprit is the : in 'Content-Type: application/json'
Sometimes using the multiline string format helps, like this:
notify_success:
stage: notify
allow_failure: true
only:
- master
script: |
curl -X POST -H 'Content-Type: application/json' --data '{"text":"New Hass config deployed successfully!"}' https://chat.bryantgeeks.com/hooks/$ROCKET_CHAT_TOKEN
Related
I'd like to have a branch in the deployment script of my .gitlab-ci.yml file that is based on whether the particular pipeline is run because a MR has been accepted and is being merged into the default branch.
For instance,
Deploy:
stage: deploy
image: alpine
script:
- apk add openssh-client
- install -d -D -m 700 -p ~/.ssh
- eval $(ssh-agent -s)
- cat "${MY_SSH_PRIVATE_KEY}" | ssh-add -
- if [ "${CI_DEFAULT_BRANCH}" = "${CI_COMMIT_BRANCH}" ]; then \
rsync $BUILD_DIR/myfile.tar.gz filestore1 ; \
else \
rsync $BUILD_DIR/myfile.tar.gz filestore2 ; \
fi
I don't think that the comparison of CI_DEFAULT* with CI_COMMIT* is a safe comparison. I believe this works anytime a commit is on the default branch, not when being merged, so I think I'm missing an important distinction and/or a best-practice.
My intent is that so long as the MR is still in dev, it should be pushed to filestore2, and only pushed to the production filestore1 when the MR is good, tests good/complete, and is accepted.
Related:
Gitlab: How to run a deploy job on a tagged version / release? requires me to tag when I need to release something; this may be something I work into my workflow, but is not our current workflow
The code you have makes sense to fulfill the purpose you described and should work as-is (assuming the bash code is otherwise well-formed).
When changes are merged to the default branch, CI_COMMIT_BRANCH will be the same as CI_DEFAULT_BRANCH and the condition you have would evaluate to true, causing it to deploy to filestore1 -- for pipelines running on any other branch, filestore2 would be used when this job runs.
Your code should be as follows, this way filestore1 will be pushed only when MR is merged to main or master, or else filestore2 will be pushed.
Deploy:
stage: deploy
image: alpine
script:
- apk add openssh-client
- install -d -D -m 700 -p ~/.ssh
- eval $(ssh-agent -s)
- cat "${MY_SSH_PRIVATE_KEY}" | ssh-add -
- if [ "${CI_COMMIT_BRANCH}" -eq "main" ] | [ "${CI_COMMIT_BRANCH}" -eq "master" ]; then \
rsync $BUILD_DIR/myfile.tar.gz filestore1 \
else \
rsync $BUILD_DIR/myfile.tar.gz filestore2 \
fi
The if statement checks for master or main here.
I have a Gitlab job in which I get a value from a .txt file. This value (v100322.1) was written into the text file in a previous stage and passing by to the job through artifacts.
When I try to get value from the file with cat command I get this error on the pipeline:
$ $PACKAGE_VERSION=$(cat build.txt)
+++ cat build.txt
++ $'=\377\376v100322.1\r'
bash: line 132: $'=\377\376v100322.1\r': command not found
And this is my YAML file for GitLab-CI:
stages:
- deploy
- trigger
.deploy_job_base:
stage: deploy
tags:
- dotnet
script:
- $PACKAGE_VERSION="v100322.1"
- ${PACKAGE_VERSION} > build.txt
artifacts:
expire_in: 1 week
paths:
- build.txt
allow_failure: false
deploy_job_sport:
extends: .deploy_job_base
deploy_job_TestClient:
extends: .deploy_job_base
# trigger GitLab API call
.trigger_base:
stage: trigger
script:
- $PACKAGE_VERSION=$(cat build.txt)
- 'curl --include --fail --request POST --form "token=$CI_JOB_TOKEN" --form "PACKAGE_VERSION=$PACKAGE_VERSION" --form "ref=feature/1000" $GITLAB_BASE_URL/api/v4/projects/$APP_PROJECT_ID/trigger/pipeline'
trigger_sport:
extends: .trigger_base
variables:
APP_PROJECT_ID: "2096"
needs: [deploy_job_sport]
dependencies:
- deploy_job_sport
trigger_TestClient:
extends: .trigger_base
variables:
APP_PROJECT_ID: "2110"
needs: [deploy_job_TestClient]
dependencies:
- deploy_job_TestClient
Do you know which is the problem here?
Thanks in advance.
The cause is the syntax, you can always check it on a virtual box, or better, pull the docker image down that the job will be executed on, and test the script in there
(using docker run -it ${JOB_DOCKER_IMAGE} /bin/bash for instance).
I just tested your script and got this:
you can clearly see bash will not like the $ in front of PACKAGE_VERSION and interpret it, like a command ...
But you can turn the script of .deploy_job_base into a one-liner like this:
you circumnavigate the need for defining a variable and just dump it in the build.txt file.
I am trying to have one job check for a word being present in a config file and have that determine whether a subsequent trigger job occurs or not...
Like so...
stages:
- check
- trigger_pipeline
variables:
- TRIGGER_JOB : "0"
- CONFIG_FILE : "default.json"
check:
stage: check
script:
- |
if grep -q keyword "$CONFIG_FILE"; then
TRIGGER_JOB="1"
fi
- echo "TRIGGER_JOB=${TRIGGER_JOB}" >> variables.env
artifacts:
reports:
dotenv: "variables.env"
trigger_pipeline:
stage: trigger_pipeline
rules:
- if: '$TRIGGER_JOB == "1"'
trigger:
project: downstream/project
branch: staging
strategy: depend
needs: ["check"]
It seems like I've reached a limitation with GitLab because the trigger_pipeline job doesn't even get created due to the fact that the pipeline initializes with TRIGGER_JOB: "0" so it doesn't matter that I'm doing this check to trigger the pipeline later if the keyword was found.
Is there any way to dynamically decide if this trigger_pipeline job would be created or not?
I would just put it all in one job and trigger the downstream pipeline through the API, but then of course, I can't depend on the downstream status which is something I want as well (and isn't possible to do when triggering through the API from everything I've found in the docs).
Any advice would be appreciated. Thanks!
The closest thing to what you're describing is dynamic child pipelines. That would allow you to create a pipeline configuration dynamically in one job, then run it.
generate-config:
stage: build
script: generate-ci-config > generated-config.yml
artifacts:
paths:
- generated-config.yml
child-pipeline:
stage: test
trigger:
include:
- artifact: generated-config.yml
job: generate-config
There is a way to dynamically choose whether or not to execute a gitlab job
In your case apply the following config
stages:
- trigger_pipeline
- check
variables:
- CONFIG_FILE : "default.json"
check:
stage: check
script:
- |
if grep -q keyword "$CONFIG_FILE"; then
curl -s --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.com/api/v4/projects/$CI_PROJECT_ID/pipelines/$CI_PIPELINE_ID/jobs" | jq '.[]'
JOB_ID=$(curl -s --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.com/api/v4/projects/$CI_PROJECT_ID/pipelines/$CI_PIPELINE_ID/jobs" | jq '.[] | select(.name=="child-pipeline") | .id')
curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.com/api/v4/projects/$CI_PROJECT_ID/jobs/$JOB_ID/play"
fi
trigger_pipeline:
stage: trigger_pipeline
trigger:
project: downstream/project
branch: staging
strategy: depend
when: manual
The logic behind is that you configure your target job, in your case trigger_pipeline as manual
Change the stage ordering so that your target job is set up first
Then if your logic evaluates to true in your case
grep -q keyword "$CONFIG_FILE"
Then the check job basically executes the target job. By firstly identifying the target job id and finally executing by calling the play endpoint of the Gitlab API
I've got a project involving multiple GitLab users, all at ownership level. I've got a gitlab-ci.yml file that creates a new tag and pushes the new tag to the repository. This was set up using a deploy key and ssh. The problem is, no matter who actually triggers the job, the same user is always listed as the triggerer, which causes some traceability problems.
Currently, the .yml looks something like this, taken from this link:
before_script:
- echo "$SSH_PRIVATE_KEY_TOOLKIT" | tr -d '\r' | ssh-add - > /dev/null
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- ssh-keyscan $GITLAB_URL >> ~/.ssh/known_hosts
- chmod 644 ~/.ssh/known_hosts
- git config --global user.email $GITLAB_USER_EMAIL
- git config --global user.name $GITLAB_USER_NAME
Where $SSH_PRIVATE_KEY_TOOLKIT is generated as suggested in the link.
! For just creating a tag, an api call would be way easier using the tags api. as the JOB TOKEN should also normally have enough permissions to do this, this would always be assigned to the executor of the job/pipeline. (untested does not work)
https://docs.gitlab.com/ee/api/tags.html#create-a-new-tag
curl --request POST --header "JOB-TOKEN: $CI_JOB_TOKEN" "$CI_API_V4_URL/projects/$CI_PROJECT_ID/repository/tags?tag_name=<tag>&ref=$CI_COMMIT_SHA"
You can always fallback to create releases with the release API, which also results in an Git Tag. https://docs.gitlab.com/ee/api/releases/index.html#create-a-release
curl --request POST --header "JOB-TOKEN: $CI_JOB_TOKEN"
--data '{ "name": "New release", "ref":"$CI_COMMIT_SHA", tag_name": "<TAG>", "description": "Super nice release"}'
"$CI_API_V4_URL/projects/$CI_PROJECT_ID/releases"
or using the the GitLab CI release directives https://docs.gitlab.com/ee/ci/yaml/index.html#release
release_job:
stage: release
image: registry.gitlab.com/gitlab-org/release-cli:latest
rules:
- if: $TAG # Run this job when a tag is created manually
script:
- echo 'running release_job'
release:
name: 'Release $TAG'
description: 'Created using the release-cli $EXTRA_DESCRIPTION' # $EXTRA_DESCRIPTION must be defined
tag_name: '$TAG' # elsewhere in the pipeline.
ref: '$TAG'
Trying to use CI/CD with VueJs with Gitlab.
Played around with 50 different gitlab-ci.yml configurations and keep having loads of issues with different stages.
I followed the following tutorial to a T:
https://about.gitlab.com/2017/09/12/vuejs-app-gitlab/
build site:
image: node:6
stage: build
script:
- npm install --progress=false
- npm run build
artifacts:
expire_in: 1 week
paths:
- dist
deploy:
image: alpine
stage: deploy
script:
- apk add --no-cache rsync openssh
- mkdir -p ~/.ssh
- echo "$SSH_PRIVATE_KEY" >> ~/.ssh/id_dsa
- chmod 600 ~/.ssh/id_dsa
- echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
- rsync -rav --delete dist/ user#server.com
I skipped the testing phase because it keeps failing...and so why not just skip it.
If it helps, with this configuration, I keep getting the following error:
What does your gitlab-ci.yml file (that works) for a VueJS webapp look like?
I bumped into this question yesterday while I was also researching how to set up CI/CD on Gitlab. After 24 hours of research and test. I finally got a working script. Hope this helps.
For this to work, you will need to:
Set up a variable STAGING_PRIVATE_KEY in Settings -> Variables section of your project
Add your ssh key to the list of know hosts on your server
Below is my final script:
image: node:latest
stages:
- build
- deploy
build site:
stage: build
before_script:
- apt-get update
- apt-get install zip unzip nodejs npm -y
- npm install --progress=false
cache:
paths:
- node_modules/
script:
- npm run build
artifacts:
expire_in: 1 week
paths:
- dist/
deploy:
stage: deploy
before_script:
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
- eval $(ssh-agent -s)
- echo "$STAGING_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- ssh-keyscan YOUR-SEVER-IP >> ~/.ssh/known_hosts
- chmod 644 ~/.ssh/known_hosts
script:
- ssh -p22 root#YOUR-SEVER-IP "mkdir /var/www/_tmp"
- scp -p22 -r /builds/YOUR-USERNAME/YOUR-REPO-TITLE/dist/* root#form.toprecng.org:/var/www/form.toprecng.org/_tmp
- ssh -p22 root#YOUR-SEVER-IP "mv /var/www/html/ /var/www/_old && mv /var/www/_tmp /var/www/html/"
- ssh -p22 root#YOUR-SEVER-IP "rm -rf /var/www/_old"